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Executive  Summary 


This  report  is  the  second  of  two  reports  concerning  how  the  Medicare  Current 
Beneficiary  Survey  (MCBS)  can  be  used  to  effectively  evaluate  the  goals  of  the  National 
Medicare  Education  Program  (NMEP).  The  report  analyzes  whether  the  NMEP  is  achieving  its 
goals  using  Rounds  23  and  24  of  the  1998  MCBS  that  were  fielded  immediately  following 
NMEP  implementation  in  the  Fall  of  1 998.  We  evaluated  MCBS  survey  questions  that  were 
explicitly  created  to  measure  progress  toward  NMEP  goals  and  can  be  used  in  future  longitudinal 
analyses.  In  the  prior  Phase  1  report,  RTI  identified  MCBS  survey  questions  that  could  be  used 
to  establish  a  pre-NMEP  baseline  as  well  as  knowledge  measures  drawn  from  a  variety  of 
projects  that  could  be  modified  and  tested  for  use  in  monitoring  progress  toward  NMEP  goals. 

Two  other  companion  reports  accompany  the  Phase  1  and  Phase  2  reports.  The 
Technical  Note  1:  Knowledge  Index  contains  psychometric  analyses  of  four  potential  measures 
that  we  considered  for  measuring  NMEP  progress.  Technical  Note  2:  Preliminary  Baseline 
Analysis  contains  the  foundation  for  longitudinal  analyses  that  take  advantage  of  the  5-state  pilot 
project  where  the  1999  Medicare  &  You  handbook  was  initially  fielded.  All  of  these  reports  seek 
to  provide  HCFA  with  information  and  recommendations  for  using  the  MCBS  to  measure 
progress  toward  the  NMEP  goals. 

The  goals  of  the  NMEP  are: 

■  increasing  beneficiary  access  to  information  on  Medicare  program  choices, 

>    raising  beneficiary  awareness  of  the  Medicare  program  and  choices  available  to  them, 
«    helping  beneficiaries  understand  the  Medicare  program  and  choices  that  can  be  made, 
and 

■  helping  beneficiaries  use  the  information  obtained  to  make  optimal  choices. 

HCFA  is  interested  in  understanding  whether  beneficiaries  have  enough  information  to 
make  informed  choices  concerning  recent  changes  in  the  Medicare  program.  These  changes 
include  increases  in  the  number  of  health  insurance  options  and  initiatives  to  promote  the  use  of 
preventive  health  services.  Rounds  23  and  24  of  the  1998  MCBS  Access  to  Care  files  contain 
questions  directly  concerning  the  1999  Medicare  &  You  Handbook  and  the  1999  Medicare  & 
You  Bulletin  that  can  be  used  to  evaluate  whether  the  NMEP  is  increasing  beneficiary  knowledge 
about  the  Medicare  program.  Progress  can  be  best  monitored  using  measures  of  beneficiary 
knowledge  as  well  as  other  measures  associated  with  one  or  more  of  the  goals  above. 
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In  order  to  achieve  this  purpose,  HCFA  and  RTI  developed  the  following  objectives  for 
the  Phase  2  report: 

•  Identify  questions  in  the  1998  MCBS  associated  with  each  of  the  four  NMEP  goals; 
■    Analyze  these  questions  using  descriptive  and  multivariate  analyses  to  determine  if 

HCFA-sponsored  materials  are  promoting  NMEP  goals;  and 

•  Draw  conclusions  and  make  technical  recommendations  based  on  these  findings. 

Scope  of  Work 

MCBS  Round  23  interviews  contained  a  supplement  of  questions  specifically  designed  to 
obtain  information  concerning  the  respondent's  need  for  and  use  of  information,  and  in 
particular,  the  1999  Medicare  &  You  Handbook  (or  Bulletin).  This  Round  23  supplement, 
known  as  the  Beneficiary  Knowledge  (BK)  Supplement,  contained  a  total  of  78  questions. 
MCBS  Round  24  interviews  contained  a  supplement  of  questions  known  as  the  Beneficiary 
Information  Needs  (BN)  Supplement  with  a  total  of  34  questions.  These  two  supplements 
together  contained  a  total  of  1 12  questions  on  various  aspects  of  beneficiary  knowledge  and 
information  needs  that  could  potentially  be  used  to  monitor  the  NMEP. 

For  each  of  the  NMEP  goals  we  identified  survey  questions  in  the  MCBS  that  could  be 
used  to  evaluate  the  effects  of  the  NMEP.  We  linked  44  different  survey  questions  in  the  MCBS 
to  policy  questions  for  determining  the  effects  of  the  NMEP  on  access  to  information,  awareness, 
understanding,  and  impact/use,  and  for  understanding  what  beneficiaries  thought  about  the  1999 
Medicare  &  You  materials  and  what  they  did  with  them. 

For  each  NMEP  goal  or  area  we  list  the  MCBS  survey  questions  analyzed  and  present  the 
results  of  both  descriptive  and  multivariate  analyses.  We  discuss  frequency  distributions  in 
narrative  form  and  present  the  results  in  bar  charts.  For  multivariate  analyses,  we  present  the 
marginal  effect  of  reading  the  1999  Medicare  &  You  handbook  in  the  five  states  in  which  it  was 
fielded.  This  main  policy  measure  is  expressed  as  the  percentage  point  difference  in  one  of  the 
1 2  survey  questions  of  interest  being  analyzed  as  a  consequence  of  reading  the  handbook  or  not. 

In  each  of  the  areas  that  we  examined  using  regression  analyses,  most  but  not  all  of  these 
survey  questions  showed  that  reading  the  handbook  made  a  positive  difference  compared  to  not 
reading  the  handbook.  One  of  the  three  access  questions,  both  of  the  awareness  questions,  all 
three  of  the  knowledge  indices,  and  two  of  the  four  impact/use  questions  showed  a  significant 
effect  of  having  read  the  handbook  on  the  outcome  measured.  In  all,  8  of  the  12  regression 
analyses  showed  that  reading  the  handbook  significantly  influenced  the  outcome.  The  results 
from  each  area  of  analysis  are  presented  below. 

Access.  We  found  that  reading  the  handbook  had  a  significant  effect  on  only  one  of  our 
three  access  questions —  whether  beneficiaries  tried  to  find  out  about  new  benefits  or  changes  in 
the  Medicare  program.  Beneficiaries  who  read  the  handbook  in  the  five  states  that  received  it 
were  only  2.8  percentage  points  more  likely  to  have  tried  to  find  out  about  new  benefits  or 
changes.  Reading  the  handbook  had  no  significant  effect  on  whether  beneficiaries  tried  to  find 
out  about  the  availability  and  benefits  of  Medicare  managed  care  plans  and  what  Medigap  or 
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supplemental  insurance  policies  cover.  In  part,  these  questions  are  not  the  best  measures  of 
access  in  that  they  only  ask  beneficiaries  whether  they  tried  to  find  information.  Better  measures 
of  access  ask  whether  beneficiaries  actually  received  information. 

Awareness.  We  found  modest  effects  of  reading  the  handbook  on  both  MCBS  questions 
that  we  used  to  measure  awareness.  Beneficiaries  who  read  the  handbook  in  the  five  states  that 
received  it  were  approximately  8  percentage  points  more  likely  to  have  been  aware  of  Medicare 
MS  As  and  8.4  percentage  points  more  likely  to  have  been  aware  of  PPOs.  Both  of  these  results 
were  statistically  significant.  We  found  in  descriptive  analyses  that  only  16%  of  beneficiaries 
were  aware  of  MSAs  and  24%  were  aware  of  PPOs,  and  even  fewer  beneficiaries  were  aware  of 
POSs  and  PSOs.  Only  9.5%  of  beneficiaries  had  heard  of  the  buy-in  programs,  and  91 .7%  of 
beneficiaries  were  unaware  that  this  type  of  help  was  available. 

Understanding.  We  also  found  modest  effects  of  reading  the  handbook  for  the  NMEP 
goal  of  understanding.  We  found  approximately  5  to  7  percentage  point  gains  in  knowledge 
scores  on  three  separate  knowledge  indices  developed  for  measuring  beneficiary  understanding 
of  different  parts  of  the  Medicare  program.  All  three  of  these  findings  were  statistically 
significant.  The  index  on  which  beneficiaries  scored  the  largest  gain  (7.3  percentage  points)  was 
the  8-item  quiz,  the  majority  of  which  contained  questions  concerning  Medicare  managed  care 
issues.  The  other  two  knowledge  indices  for  which  reading  the  handbook  in  the  five  states  had 
an  effect  were  the  Perceived  Medicare  Knowledge  index  and  the  3-item  quiz,  which  showed 
percentage  point  increases  of  5.9  and  4.8  respectively. 

Impact/Use.  We  also  measured  the  impact  and  use  of  the  handbook  in  the  five  states, 
finding  significant  results  in  two  of  four  regression  models  we  analyzed.  Beneficiaries  who  read 
the  handbook  were  16.7  percentage  points  more  likely  to  have  ever  used  the  handbook  to  find 
information  about  health  plan  options,  and  12.2  percentage  points  more  likely  to  have  reviewed 
information  about  different  Medicare  health  plan  options.  We  did  not  find  a  significant  effect  for 
having  read  the  handbook  in  satisfaction  or  confidence  measures  we  analyzed.  Beneficiaries 
were  no  more  likely  to  have  been  satisfied  with  the  availability  of  information  about  the 
Medicare  program  when  they  needed  it,  nor  were  they  more  confident  that  they  could  choose  a 
health  care  plan  that  best  matched  their  needs. 

Direct  Evaluation  of  NMEP  Materials.  Beneficiaries  who  get  the  handbook  or  bulletin 
materials  read  them  and  find  them  fairly  easy  to  understand,  but  only  a  minority  of  beneficiaries 
report  receiving  these  materials.  Only  26.4%  of  all  respondents  reported  reading  all  or  parts  of 
the  1999  handbook.  We  found  that  sizeable  majorities  of  beneficiaries  reported  the  materials  in 
general  and  a  comparison  chart  within  them  were  either  very  easy  or  somewhat  easy  to 
understand,  but  almost  15%  of  beneficiaries  said  that  they  were  difficult  to  understand. 

Conclusion 

Reading  the  handbook  shows  potential  for  increasing  knowledge  of  Medicare 
beneficiaries,  raising  awareness  of  managed  care  arrangements  and  buy-in  programs,  and  getting 
beneficiaries  to  review  health  plan  choices.  Reading  the  handbook  in  the  five  states  had  modest 
percentage  point  gains  for  the  NMEP  goal  on  impact/use  as  beneficiaries  used  it  to  find  or  review 
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information  on  health  plan  choices.  When  considering  the  effects  of  reading  the  handbook  on 
satisfaction  and  confidence  issues,  beneficiaries  who  read  the  handbook  may  find  that  their 
choices  and  options  are  more  complicated  and  numerous  than  previously  thought,  and  are  not 
necessarily  more  satisfied  about  information  from  the  Medicare  program  or  confident  in  making 
health  plan  choices. 

We  found  Rounds  23  and  24  of  the  MCBS  to  be  generally  of  benefit  for  evaluating  the 
NMEP,  but  several  improvements  are  needed  for  comprehensive  monitoring.  In  general,  more 
MCBS  questions  are  needed  to  better  measure  the  four  NMEP  goals  (see  our  Phase  1  report, 
Chapter  6).  We  found  some  survey  questions  in  the  MCBS  that  could  be  used  to  evaluate  each 
of  the  goals  of  the  NMEP,  though  some  were  better  than  others. 

Our  findings  suggest  the  importance  of  ongoing  development,  testing,  and  refinement  of 
knowledge  indices  as  an  important  way  to  measure  beneficiary  knowledge  and  limit  the  impact 
of  social  desirability  in  responses.  Additional  survey  questions,  especially  questions  with 
objective  answers,  are  needed  for  knowledge  index  development.  In  addition,  HCFA  should 
continue  to  identify  better  measures  of  access  to  information  that  more  effectively  measure  the 
influence  of  the  NMEP.  A  consistent  set  of  questions  will  need  to  be  administered  over  time  in 
order  to  assess  the  longer  range  effects  of  the  NMEP.  Over  time,  there  may  be  diminishing 
returns  to  knowledge  improvement  as  a  consequence  of  reading  the  handbook  or  bulletin. 

Though  a  minority  of  beneficiaries  report  receiving  the  handbook,  most  beneficiaries  who 
get  the  handbook  read  it,  and  most  keep  it  for  future  reference.  Beneficiaries  more  likely  to  be 
reading  the  handbook  are  those  who  may  already  be  making  informed  choices  as  opposed  to 
those  who  need  more  help  in  doing  so.  This  finding  may  need  to  be  taken  into  consideration 
when  interpreting  the  marginal  effects  of  reading  the  handbook  presented  in  this  report.  In 
general,  HCFA  should  continue  to  fund  a  diverse  array  of  measures  to  educate  beneficiaries. 
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1.0  Introduction 


1.1     Background  on  NMEP 

Recent  changes  to  the  Medicare  program  increase  the  number  and  range  of  options  open 
to  individual  beneficiary  choice.  Understanding  these  options  and  choosing  optimally  places  a 
greater  burden  on  beneficiaries  at  the  same  time  that  they  express  concern  over  decreasing 
cognitive  acuity  from  aging  (Gibbs  et  al.,  1996).  Thus,  the  need  is  greater  than  ever  for  easily 
understood  information  about  Medicare  that  beneficiaries  are  motivated  to  use. 

Several  studies  have  documented  that  adults  of  all  ages  have  an  inadequate  understanding 
of  their  health  insurance  coverage  (Mechanic,  1989;  Isaacs,  1996;  Garnick,  1993).  Now  there  is 
a  growing  body  of  research  documenting  the  low  level  of  understanding  that  Medicare 
beneficiaries  possess  about  the  Medicare  program  (Gibbs  et  al.,  1996;  Hibbard  et  al.,  1998; 
Murray  and  Shatto,  1998;  National  Academy  of  Social  Insurance  (NASI),  1998).  Many 
beneficiaries  do  not  understand  what  is  covered,  who  operates  the  program,  or  what  plan  options 
are  available  under  the  Medicare  program.  Many  have  never  heard  of  a  Medicare  HMO  and  the 
vast  majority  are  unable  to  identify  the  differences  between  the  original  Medicare  program  and  a 
Medicare  managed  care  plan  (Hibbard  et  al.,  1998).  This  is  an  especially  important  finding,  as 
understanding  the  differences  among  original  Medicare,  Medicare  HMOs,  and  the  newer 
Medicare+Choice  options  is  a  prerequisite  to  making  an  informed  choice. 

The  National  Medicare  Education  Program  (NMEP)  is  the  Health  Care  Financing 
Administration's  (HCFA)  coordinated  effort  to  address  this  problem,  in  both  the  short  and  long 
term,  by  creating  useful  information  resources  (e.g.,  print  materials,  Internet  materials,  toll-free 
information  hotlines,  trained  intermediaries),  informing  beneficiaries  that  these  resources  are 
available,  and  motivating  them  to  use  the  resources. 

The  goals  of  the  NMEP  are: 

•  increasing  beneficiary  access  to  information  on  Medicare  program  choices, 

raising  beneficiary  awareness  of  the  Medicare  program  and  choices  available  to  them, 
■    helping  beneficiaries  understand  the  Medicare  program  and  choices  that  can  be  made, 
and 

•  helping  beneficiaries  use  the  information  obtained  to  make  optimal  choices. 

HCFA  is  interested  in  identifying  and  using  data  collected  from  beneficiaries  to  monitor 
progress  toward  the  NMEP  goals.  Progress  can  be  best  monitored  using  measures  of  beneficiary 
knowledge  as  well  as  other  measures  associated  with  one  or  more  of  the  goals  above.  Changes 
in  these  measures  over  time  will  be  the  best  indicators  of  progress  toward  meeting  these  goals. 
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1.2  Purpose 


The  purpose  of  this  report  is  to  monitor  progress  toward  achievement  of  each  of  the  four 
NMEP  goals  using  questions  drawn  from  Rounds  23  and  24  of  the  1998  Medicare  Current 
Beneficiary  Survey  (MCBS)  Access  to  Care  files.  These  files  contain  questions  directly 
concerning  the  1999  Medicare  &  You  Handbook  and  the  1999  Medicare  &  You  Bulletin  that  can 
be  used  to  evaluate  whether  the  NMEP  is  increasing  beneficiary  knowledge.  HCFA  is  interested 
in  understanding  whether  beneficiaries  have  enough  information  to  make  informed  choices 
concerning  recent  changes  in  the  Medicare  program.  These  changes  include  increases  in  the 
number  of  health  insurance  options,  availability  of  buy-in  programs,  and  initiatives  to  promote 
the  use  of  preventive  health  services. 

In  order  to  achieve  this  purpose,  HCFA  and  RTI  have  developed  the  following 
objectives: 

■  Identify  questions  in  the  1998  MCBS  associated  with  each  of  the  four  NMEP  goals; 

■  Analyze  these  questions  using  descriptive  and  multivariate  analyses  to  determine  if 
HCFA-sponsored  materials  are  promoting  NMEP  goals;  and 

■  Draw  conclusions  and  make  policy  recommendations  based  on  these  findings. 

This  report  is  the  second  of  two  reports  concerning  how  the  MCBS  can  be  used  to 
measure  progress  toward  NMEP  goals.  This  Phase  2  report  will  evaluate  progress  toward  NMEP 
goals  using  1998  MCBS  Rounds  23  and  24  which  were  fielded  immediately  following  NMEP 
implementation.  The  Phase  2  report  contains  analyses  using  MCBS  survey  questions  that  were 
explicitly  created  to  measure  progress  toward  NMEP  goals  and  can  be  used  in  future  longitudinal 
analyses.  In  the  prior  Phase  1  report,  RTI  identified  MCBS  survey  questions  that  could  be  used 
to  establish  a  pre-NMEP  baseline  as  well  as  knowledge  measures  drawn  from  a  variety  of 
projects  that  could  be  modified  and  tested  for  use  in  monitoring  progress  toward  NMEP  goals. 

Two  other  companion  reports  accompany  the  Phase  1  and  Phase  2  reports.  The 
Technical  Note  1:  Knowledge  Index  contains  psychometric  analyses  of  four  potential  measures 
that  we  considered  for  measuring  NMEP  progress.  Technical  Note  2:  Preliminary  Baseline 
Analysis  contains  the  foundation  for  longitudinal  analyses  that  takes  advantage  of  the  5-state 
pilot  project  where  the  1999  Medicare  &  You  handbook  was  initially  fielded.  All  of  these 
reports  seek  to  provide  HCFA  with  information  and  recommendations  for  using  the  MCBS  to 
measure  progress  toward  the  NMEP  goals. 

2.0    Description  of  1998  MCBS  and  Questions  Used  for  Monitoring 
the  NMEP 

We  describe  in  this  section  how  the  MCBS  is  related  to  the  NMEP  implementation  and 
goals,  and  how  we  identified  MCBS  questions  appropriate  for  the  evaluation  of  the  NMEP. 
Survey  questions  were  drawn  from  several  rounds  of  the  1998  MCBS  and  linked  to  policy 
questions  we  sought  to  answer  about  how  well  the  NMEP  met  its  goals.  We  also  describe  the 
types  of  measures  we  created  using  these  survey  questions  to  measure  knowledge. 
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2.1 


Description  of  1998  MCBS  As  Related  to  the  NMEP 


The  1998  MCBS  was  fielded  in  three  rounds  of  interviews  beginning  in  September  1998. 
The  three  time  periods  over  which  interviews  were  conducted  were: 

Round  22  from  September  1998  to  December  1998; 
Round  23  from  January  1999  to  April  1999;  and 
Round  24  from  May  1999  to  August  1999. 

The  1999  Medicare  &  You  Handbook  was  initially  mailed  in  the  Fall  of  1998  in  five 
states  (Oregon,  Washington,  Ohio,  Florida,  and  Arizona)  as  a  pilot  study.  This  pilot  study  of  the 
handbook  occurred  during  the  fielding  of  the  Round  22  (Fall  1998)  MCBS  interviews,  allowing 
HCFA  to  develop  specific  questions  related  to  the  use  of  the  handbook  in  Rounds  23  and  24. 
These  same  questions  with  minor  wording  changes  were  also  administered  to  respondents  in  the 
remaining  45  states  who  had  received  the  Medicare  and  You  1999  Bulletin  during  1998. 

MCBS  Round  23  interviews  contained  a  supplement  of  questions  specifically  designed  to 
obtain  information  concerning  the  respondent's  need  for  and  use  of  information,  and  in 
particular,  the  1999  Medicare  &  You  Handbook  (or  Bulletin).  This  Round  23  supplement, 
known  as  the  Beneficiary  Knowledge  (BK)  Supplement,  contained  a  total  of  78  questions. 
MCBS  Round  24  interviews  contained  a  supplement  of  questions  known  as  the  Beneficiary 
Information  Needs  (BN)  Supplement  with  a  total  of  34  questions.  These  two  supplements 
together  contained  a  total  of  1 12  questions  on  various  aspects  of  beneficiary  knowledge  and 
information  needs  which  could  be  used  to  monitor  the  NMEP. 

2.2     MCBS  Measures  Used  to  Evaluate  Achievement  of  NMEP  Goals 

RTI  proposed  a  series  of  policy  questions  designed  to  monitor  progress  toward  attainment 
of  each  of  the  four  NMEP  goals  in  its  Final  Analysis  Plan  dated  August  19,  1999.  Each  of  the 
MCBS  questions  was  linked  to  one  of  these  policy  questions.  Some  MCBS  questions  could 
arguably  be  associated  with  more  than  one  NMEP  goal,  but  in  the  report  we  linked  each  question 
with  the  goal  that  we  believed  it  most  strongly  reflected. 

After  reviewing  the  descriptive  frequencies  of  each  question  in  the  MCBS  data,  we 
selected  those  questions  that  had  sample  sizes  large  enough  to  potentially  answer  each  policy 
question  and  provide  a  broad  picture  of  the  effects  of  the  NMEP  on  beneficiary  knowledge  and 
choices.  We  were  able  to  identify  enough  questions  to  give  an  assessment  of  how  well  the 
NMEP  is  attaining  each  of  its  four  goals.  The  individual  questions  that  we  have  chosen  to 
present  in  the  analysis  section  of  this  report  appear  in  Section  4  grouped  by  the  four  NMEP 
goals. 
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One  way  to  measure  progress  toward  NMEP  goals  is  by  measuring  beneficiaries' 
knowledge  of  the  Medicare  program.  In  addition  to  identifying  the  individual  MCBS  questions 
above  for  analysis,  we  developed  three  multi-item  knowledge  indices  using  most  of  the 
individual  MCBS  questions  from  the  NMEP  goal  on  Understanding.  These  knowledge  indices 
are  designed  to  measure  achievement  of  NMEP  goals  in  a  more  comprehensive  way  than  can  be 
achieved  by  analyzing  any  single  question  alone. 

In  our  report  entitled  Technical  Note  1:  Knowledge  Index,  we  analyzed  data  for  these 
three  knowledge  indices  to  provide  estimates  of  their  internal  consistency  reliability  and 
construct  validity.  These  two  characteristics  are  important  in  determining  whether  scores  yielded 
by  a  knowledge  index  are  repeatable  and  whether  they  provide  important  information  about 
beneficiary  knowledge. 

The  three  knowledge  indices  used  in  this  report  to  monitor  the  NMEP  are  the  8-item  quiz, 
the  Perceived  Medicare  Knowledge  Index,  and  the  3 -item  quiz.  The  8-item  quiz  and  the 
Perceived  Medicare  Knowledge  Index  were  found  to  be  both  internally  consistent  and  valid 
constructs.  We  determined  that  the  3 -item  quiz  was  a  valid  measure  of  knowledge  but  it  did  not 
have  enough  questions  to  be  deemed  internally  consistent.  The  addition  of  more  questions  to  the 
3-item  index  would  improve  its  internal  consistency.  Still,  for  this  Phase  2  report,  we  have 
decided  to  include  the  3-item  quiz  as  a  knowledge  index  and  present  results  for  it  so  that  we 
could  view  any  differences  in  its  performance  compared  to  the  8-item  quiz  and  the  Perceived 
Medicare  Knowledge  Index. 

3.0  Description  of  Analytic  Methods 

3.1  Descriptive  Analyses 

To  investigate  the  potential  for  the  NMEP  to  have  an  effect  on  beneficiary  knowledge 
and  other  measures,  we  performed  analyses  of  the  MCBS  questions  grouped  by  five  areas — the 
four  NMEP  goals  (Access,  Awareness,  Understanding,  and  Impact/Use)  and  a  fifth  group  of 
questions  concerning  a  direct  evaluation  of  the  handbook  and  bulletin  materials.  We  performed 
all  analyses  using  the  sampling  weights  and  SUDAAN  software  which  take  into  account  the 
complex  survey  design  in  order  to  obtain  reliable  estimates  for  the  entire  Medicare  population. 
Weighted  sample  sizes  for  all  analyses  ranged  from  648,650  to  30,105,201. 

First,  we  performed  frequency  distributions  for  all  questions  individually.  We  reported 
these  results  in  bar  charts  in  the  appendices  for  Exhibits  B1-B7  for  Access  questions,  Exhibits 
C1-C4  for  Awareness  questions,  Exhibits  D1-D24  for  Understanding  questions,  Exhibits  E1-E5 
for  Impact/Use  questions,  and  Exhibits  F1-F4  for  questions  concerning  the  direct  evaluation  of 
the  handbook  or  bulletin  materials.  These  exhibits  are  found  in  the  back  of  the  report  in 
Appendices  B-F.  We  briefly  report  on  these  results  in  narrative  form  in  Section  4  grouped  by 
each  of  these  five  areas. 

Second,  we  performed  bivariate  analyses  on  each  of  the  three  knowledge  indices  (the  8- 
item  quiz,  the  Perceived  Medicare  Knowledge  Index,  and  the  3-item  quiz)  to  explore  how 
knowledge  varies  across  certain  beneficiary  characteristics  selected  to  represent  a  wide  range  of 
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beneficiary  experiences.  We  identified  eight  beneficiary  characteristics  for  comparisons  in  the 
bivariate  analyses.  We  selected  four  demographic  and  socio-economic  measures  (gender,  age 
group,  race,  and  education),  one  health  status  indicator  (self-reported  health  status),  two  health 
insurance  indicators  (the  type  of  insurance  held  and  whether  a  beneficiary  had  any  managed  care 
experience),  and  one  indicator  of  the  extent  of  health  services  utilization  (total  Medicare 
charges). 

We  present  the  distribution  of  knowledge  scores  across  these  beneficiary  characteristics 
without  the  usual  tests  of  association  often  used  in  bivariate  analyses  because  sample  sizes  in  the 
MCBS  are  so  large  that  tests  of  association  are  usually  significant  at  p<0.01  (even  for  small 
differences  in  mean  knowledge  scores  within  a  beneficiary  characteristic).  Instead,  we  have 
chosen  to  limit  significance  testing  to  our  multivariate  analyses  where  we  can  control  for  the 
independent  effect  that  each  of  these  beneficiary  characteristics  has  on  the  knowledge  index 
being  tested.  These  bivariate  distributions  are  nonetheless  informative  for  understanding  the 
distribution  of  knowledge  within  each  beneficiary  characteristic. 

Various  studies  have  demonstrated  differences  between  variables  associated  with  these 
indicators  and  knowledge  of  Medicare  beneficiaries  about  their  insurance.  Demographic  and 
socio-economic  factors  commonly  associated  with  higher  levels  of  knowledge  include  being 
male  (Lambert,  1980),  younger  age  (Lambert,  1980;  Caffereta,  1984;  McCall,  Rice  and  Sangl, 
1986),  being  white  (Marquis,  1983;  McCall,  Rice  and  Sangl,  1986),  and  having  higher  education 
(Caffereta,  1984;  McCall,  Rice,  and  Sangl,  1986;  Hibbard,  et  al.,  1998;  Lambert,  1980;  Marquis, 
1983;  McCall,  Rice,  and  Sangl,  1986). 

Although  self-reported  health  status  has  not  been  previously  found  in  the  literature  to  be  a 
significant  predictor  of  knowledge,  we  explored  this  relationship  using  the  MCBS  data.  We 
found  self-reported  health  status  to  be  a  significant  predictor  of  knowledge  in  our  Phase  1  report 
on  baseline  MCBS  characteristics  that  could  be  used  to  monitor  the  NMEP,  and  therefore 
included  this  indicator  in  our  bivariate  analyses. 

Several  studies  have  explored  the  effect  of  types  of  insurance  coverage  on  knowledge. 
Marquis  (1983)  found  knowledge  to  be  significantly  associated  with  being  offered  a  choice  of 
health  plans  and  adversely  affected  by  plan  complexity.  Caffereta  (1984)  found  that  having 
supplemental  insurance  was  positively  related  to  knowledge.  She  also  found  that  the  use  of 
services  was  positively  associated  with  knowledge  in  a  model  subset  to  the  privately  insured, 
while  other  studies  have  not  found  an  effect. 

3.2     Multivariate  Analyses 

3.2.1  Dependent  Variables 

For  our  multivariate  analysis,  we  created  12  dependent  variables  using  MCBS  survey 
questions.  Of  the  12  models,  three  represent  the  goal  of  Access,  two  reflect  Awareness,  three 
reflect  Understanding,  and  four  measure  Impact/Use.  Three  of  these  12  models  are  Ordinary 
Least  Squares  regressions  on  the  three  knowledge  indices  we  have  constructed  to  measure  the 
NMEP  effects  on  the  goal  of  Understanding.  The  remaining  nine  models  use  categorical 


5 


measures  of  access  to  information,  awareness,  or  impact/use  as  dependent  variables  in  either 
logistic  or  ordered  logistic  regressions.  The  measures  chosen  for  dependent  variables  in  the  12 
multivariate  analyses  are  described  in  Section  4,  and  bar  charts  presenting  the  frequency 
distributions  of  these  variables  are  presented  with  related  bar  charts  for  each  NMEP  goal. 

3.2.2  Independent  Variables 

We  identified  a  set  of  17  independent  variables  drawn  from  measures  in  the  MCBS  to  use 
in  multivariate  analyses.  One  of  these  independent  dichotomous  variables  — whether  a 
respondent  had  read  the  handbook  or  bulletin  materials  thoroughly  or  in  part,  as  opposed  to  not 
reading  the  materials —  reflects  the  principal  effect  of  the  NMEP  on  each  of  the  dependent 
variables  in  the  analyses.  The  remaining  independent  variables  are  used  to  control  for  other 
factors  that  may  influence  the  respective  outcomes  being  measured.  Bar  charts  presenting  the 
frequency  distributions  of  these  independent  variables  are  presented  as  Exhibits  G1-G15. 

To  separate  the  effects  of  reading  the  1999  Medicare  &  You  Handbook  from  the  effects 
of  reading  the  1999  Medicare  &  You  Bulletin,  we  interacted  the  variable  that  reflects  reading 
these  materials  with  a  dichotomous  variable  indicating  whether  the  respondent  was  in  one  of  the 
five  states  that  received  the  Handbook  during  the  pilot  study.  After  performing  each  regression, 
we  predicted  the  marginal  effect  of  having  read  the  1999  Medicare  &  You  Handbook  in  the  five 
states  where  the  handbook  was  fielded.  We  calculated  the  marginal  effect  by  taking  the 
difference  of  reading  and  not  reading  the  handbook  in  the  five  states  and  we  report  this  result  in 
the  narrative  section  of  each  NMEP  goal  analyzed.  It  is  expressed  as  a  percentage  point 
increase/decrease  in  the  dependent  variable  for  having  read  the  handbook  in  the  five  states 
compared  to  those  in  the  five  states  who  did  not  read  it.  For  each  regression,  we  report  whether 
the  marginal  effect  of  reading  the  handbook  was  significant  as  well  as  whether  other  variables 
employed  as  controls  in  the  analyses  were  significant. 

We  created  several  dichotomous  and  categorical  variables  to  control  for  socioeconomic 
and  demographic  effects  not  related  to  reading  the  handbook  or  bulletin  materials.  We  identified 
whether  respondents  were  serving  as  proxies  for  the  sample  person  with  sample  beneficiaries  as 
the  reference  group.  We  also  separately  identified  who  made  health  insurance  decisions  for  each 
respondent.  A  categorical  variable  was  constructed  with  three  levels:  sample  beneficiaries  who 
make  their  own  health  care  decisions,  sample  beneficiaries  who  receive  help  in  making  their 
health  care  decisions,  and  sample  person  for  whom  someone  else  makes  their  health  care 
decisions  as  the  reference  group. 

We  created  a  dichotomous  variable  for  whether  beneficiaries  had  been  enrolled  in 
managed  care  with  beneficiaries  who  had  not  been  as  the  reference  group.  To  describe  health 
insurance  status,  we  created  a  categorical  variable  grouped  as  respondents  with  Medicare  only  as 
the  reference  group,  respondents  with  Medicare  and  supplemental  insurance  such  as  Medigap 
policies,  and  respondents  with  Medicare  and  Medicaid.  To  describe  levels  of  health  care 
utilization,  we  created  a  categorical  variable  representing  the  respondent's  total  Medicare 
charges  for  the  year. 
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We  also  included  independent  variables  describing  the  types  of  media  available  to 
respondents.  We  created  one  dichotomous  variable  for  whether  beneficiaries  had  cable  TV  or 
not,  and  another  dichotomous  variable  for  whether  they  had  a  personal  computer  or  not.  We  also 
created  a  categorical  variable  for  whether  respondents  had  access  to  the  Internet,  with  levels  of 
having  access,  having  never  heard  of  the  Internet,  and  finally  not  having  access  to  the  Internet  as 
the  reference  group. 

For  all  multivariate  analyses  except  for  the  three  using  the  knowledge  indices  as 
dependent  variables,  we  included  the  Perceived  Medicare  Knowledge  index  as  an  independent 
variable  because  respondents  who  think  they  have  varying  levels  of  perceived  Medicare 
knowledge  may  act  differently  when  making  choices.  We  did  not  include  the  Perceived 
Medicare  knowledge  index  in  the  other  knowledge  index  regressions  because  it  would  be  highly 
correlated  with  the  dependent  variables. 

3.2.3  Limitations  in  Estimation 

Three  types  of  endogeneity  may  be  present  in  some  or  all  of  the  multivariate  analyses 
performed.  In  particular,  our  estimations  measure  the  effect  of  reading  the  handbook  on  any  of 
12  dependent  variables.  If  these  dependent  variables  contribute  to  beneficiaries  reading  the 
handbook,  implying  reverse  causality,  then  endogeneity  may  result,  producing  biased  estimates 
of  reading  the  handbook  on  the  dependent  variable.  Other  sources  of  endogeneity  include  the 
case  where  beneficiaries  with  certain  characteristics  are  more  likely  to  read  the  handbook  than 
other  beneficiaries  (selection  bias),  and  the  case  where  a  beneficiary  performs  the  activity  being 
measured  prior  to  reading  the  handbook  rather  than  after  reading  it.  As  traditional  corrections  for 
endogeneity  always  result  in  larger  variance  estimates,  no  corrections  for  it  were  made  in  these 
analyses. 

4.0  Results 

For  each  NMEP  goal  or  area  we  list  the  survey  questions  analyzed  and  present  the  results 
of  both  descriptive  and  multivariate  analyses.  We  discuss  frequency  distributions  in  narrative 
form  and  present  the  results  in  either  bar  charts  or  frequency  tables  in  exhibits  at  the  end  of  the 
entire  report.  For  multivariate  analyses,  we  present  the  marginal  effect  of  reading  the  7999 
Medicare  &  You  handbook  as  a  percentage  difference  in  the  outcome  being  measured  and 
discuss  only  the  remaining  results  that  were  found  to  be  significant.  We  present  the  regression 
results  in  tables  in  the  back  of  the  report. 

The  main  policy  measure  of  interest  in  the  multivariate  analyses  is  the  effect  that  reading 
the  handbook  has  in  the  five  states  that  received  it  on  one  of  the  12  outcomes  of  interest  being 
analyzed.  To  understand  more  fully  who  reported  reading  either  the  handbook  or  the  bulletin 
and  who  did  not,  we  performed  a  logistic  regression  on  this  variable  with  all  of  the  independent 
variables  to  be  used  in  our  analyses.  These  results  are  presented  in  Exhibit  Al  in  Appendix  A. 
Beneficiaries  who  were  more  likely  to  be  reading  the  handbook  or  the  bulletin: 

■    lived  in  the  five  states  that  received  the  handbook  in  1 998  rather  than  in  the  45  states 
that  received  only  the  bulletin; 
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■  were  male; 

■  had  either  high  school  or  college  educations; 

■  made  their  own  health  care  decisions  or  had  help  in  doing  so  as  opposed  to  having 
someone  else  make  these  decisions  for  them; 

■  had  supplemental  insurance  such  as  a  Medigap  policy  as  opposed  to  having  Medicare 
alone;  and 

■  had  higher  scores  on  the  Perceived  Medicare  Knowledge  index. 

Most  of  these  factors  are  similar  to  those  reported  in  the  research  literature  for  Medicare 
beneficiaries  with  higher  knowledge  scores.  When  taking  these  findings  together,  the  results 
suggest  that  beneficiaries  more  likely  to  be  reading  the  handbook  are  those  beneficiaries  who 
may  already  be  making  informed  choices  as  opposed  to  those  who  need  more  help  in  doing  so. 
This  finding  may  need  to  be  taken  into  consideration  when  interpreting  the  marginal  effect  of 
reading  the  handbook. 

4.1  Access 

We  analyzed  seven  questions  that  reflect  the  NMEP  goal  of  Access  to  Information. 
These  questions  and  the  policy  research  questions  they  address  are  listed  in  Exhibit  1. 
Descriptive  frequencies  for  each  of  the  MCBS  survey  questions  are  shown  in  bar  charts  in  the 
Appendix  in  Exhibits  B1-B7. 


Exhibit  1.  Access  Research  Questions  and  Related  MCBS  Questions 


Research  Questions 

Relevant  MCBS  Survey  Items 

I.  Access 

What  information  about  Medicare  do  beneficiaries 
access  after  NMEP  implementation? 

What  types  of  information  have 
beneficiaries  tried  to  access? 

BK3  -  In  the  past  year,  have  you  tried  to  find  out  how  much  you  needed 
to  pay  for  a  particular  medical  service? 

BK7  -  In  the  past  year,  have  you  tried  to  find  information  about  new 
benefits  or  changes  in  the  Medicare  program? 

BK 1 1  -  In  the  past  year,  have  you  tried  to  find  information  about  what 

medical  services  Medicare  covers  and  does  not  cover? 

BK15  -  In  the  past  year,  have  you  tried  to  find  out  about  the  availability 

and  benefits  of  Medicare  managed  care  plans,  such  as  HMOs? 

BK19  -  In  the  past  year,  have  you  tried  to  find  information  about  what 

your  Medigap  or  supplemental  insurance  policy  covers? 

BK23  -  In  the  past  year,  have  you  tried  to  find  information  about  your 

Medicaid  plan,  such  as  how  it  works  with  Medicare? 

How  trustworthy  are  beneficiaries  of 
information  gleaned  from  radio  and 
television? 

BK52  -  You  can't  really  trust  what  you  see  on  television  or  read  in  the 
newspapers  about  Medicare.  Is  that  mostly  true  for  you,  mostly  false  for 
you,  or  neither  true  or  false? 

Six  of  these  seven  questions  (BK  3,  7, 1 1, 15, 19,  and  23)  concern  the  types  of 
information  to  which  respondents  have  tried  to  gain  access  to  information.  Overall,  the 
percentage  of  respondents  who  attempted  to  gain  information  regarding  any  of  the  topics  in  these 
six  questions  ranged  from  only  5-12%.  Too  few  sample  members  (less  than  approximately  100 
in  each  case)  responded  to  the  follow-up  question  about  which  sources  were  found  to  be  most 
helpful  to  enable  meaningful  analyses  of  questions  beyond  the  initial  screening  question  listed 
above. 
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These  descriptive  results  suggest  that  few  Medicare  beneficiaries  actively  attempt  to  find 
information  regarding  their  health  insurance  issues.  It  is  discouraging  that  such  a  small 
percentage  of  respondents  sought  information  regarding  these  six  topics  that  seem  central  to  the 
function  of  the  Medicare  program.  Beneficiaries  would  seemingly  need  information  concerning 
particular  medical  services,  new  benefits  or  changes  in  benefits,  which  services  are  covered, 
availability  and  benefits  of  managed  care  plans,  Medigap  plans  and  Medicaid  issues  in  order  to 
make  informed  choices.  Either  most  respondents  are  making  uninformed  choices,  or  are  not 
making  choices  and  accepting  by  default  what  comes  their  way,  or  perhaps  relying  on  someone 
else  to  get  the  information  for  them.  On  the  other  hand,  these  questions  may  not  be  ideal  for 
identifying  whether  the  NMEP  is  achieving  its  goal  of  access  to  information. 

The  last  MCBS  question  (BK52)  in  the  exhibit  above  concerns  whether  beneficiaries  trust 
what  is  described  about  Medicare  on  television  and  in  newspapers.  Respondents  were  asked  to 
respond  to  a  statement  that  "they  can't  really  trust  what  they  see  on  television  and  in  newspapers 
about  Medicare"  with  a  true/false  response.  Approximately  42%  of  respondents  indicated  that 
they  don't  trust  what  these  media  say  about  Medicare,  while  23%  indicated  that  they  could  trust 
what  they  see  on  television  and  in  newspapers.  One-third  of  respondents  said  the  statement  was 
neither  true  nor  false,  with  only  2%  stating  that  they  did  not  know.  These  respondents  do  have 
an  opinion  about  what  they  are  seeing  and  hearing,  but  greatly  differ  on  whether  one  can  trust 
information  about  Medicare  presented  in  these  media.  Both  good  and  bad  information  about 
Medicare  is  presented  in  both  media  in  the  forms  of  reports  from  government  and  the  Congress, 
political  advertising,  investigative  reporting  by  both  newspapers  and  independent  government 
agencies.  Given  that  less  than  one-quarter  of  respondents  indicated  that  they  mostly  trusted  what 
appears  in  these  media  about  Medicare,  they  may  be  accessing  information  from  other  sources 
that  they  trust  more. 

For  multivariate  analyses,  we  identified  three  of  the  seven  questions  from  the  exhibit 
above  that  described  areas  with  the  most  change  in  the  Medicare  program  in  recent  years.  These 
three  measures  were  the  best  available  to  us  for  investigation  of  the  effects  of  the  NMEP  on 
Medicare  beneficiary  access  to  information.  The  three  questions  used  were: 

■  In  the  past  year,  have  you  tried  to  find  information  about  new  benefits  or  changes  in 
the  Medicare  program?  (BK7) 

■  In  the  past  year,  have  you  tried  to  find  out  about  the  availability  and  benefits  of 
Medicare  managed  care  plans,  such  as  HMOs?  (BK15) 

■  In  the  past  year,  have  you  tried  to  find  information  about  what  your  Medigap  or 
supplemental  insurance  policy  covers?  (BK19) 

We  performed  three  logistic  regressions  using  each  of  these  three  questions  as  dependent 
variables  to  analyze  factors  that  predict  access  to  information.  Results  of  these  three  regressions 
are  presented  in  Exhibit  A2  in  Appendix  A.  These  results  show  that  most  of  the  Medicare 
characteristics  modeled  had  a  significant  effect  on  the  responses  for  each  question. 

We  calculated  the  marginal  effect  of  having  read  the  Medicare  and  You  handbook  in  the 
five  states  that  received  it  for  each  of  the  three  questions  modeled  above.  As  can  be  seen  in 
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Exhibit  2,  beneficiaries  who  had  read  the  handbook  in  the  five  states  were  2.8  percentage  points 
more  likely  to  have  tried  to  find  information  about  new  benefits  or  changes  in  Medicare  (about 
7.5%  of  beneficiaries  who  read  the  handbook  had  tried  to  find  information  about  new  benefits  or 
changes  in  Medicare  compared  to  only  about  4.7%  of  beneficiaries  who  did  not  read  the 
handbook).  The  effect  of  reading  the  handbook  was  not  significant  for  beneficiaries  trying  to 
find  out  about  the  availability  and  benefits  of  Medicare  managed  care  plans,  nor  for  beneficiaries 
trying  to  find  information  about  what  their  Medigap  or  supplemental  insurance  policy  covers. 


Exhibit  2.  Marginal  Effect  of  Reading  the  Handbook  in  the  Five  Intervention  States 


MCBS  Question 

Marginal  effect 

Tried  to  find  information  about  new  benefits  or  changes  in 
Medicare 

2.81  pet.  points** 

Tried  to  find  out  about  the  availability  and  benefits  of  Medicare 
managed  care  plans,  such  as  HMOs 

0.86  pet.  points 

Tried  to  find  information  about  what  your  Medigap  or 
supplemental  insurance  policy  covers 

1 .25  pet. points 

**  Significant  at  p<0.01 


These  small  effects  may  be  the  result  of  several  factors.  These  questions  are  not  the  best 
measures  of  access  to  information.  The  research  literature  often  uses  measures  that  describe 
whether  someone  actually  received  the  service  or  information  in  question.  These  measures  only 
ask  if  someone  tried  to  find  information.  Nevertheless,  if  better  questions  to  describe  access  to 
information  could  be  found  in  the  MCBS,  they  may  still  show  only  modest  improvements  of  the 
effect  of  the  NMEP  on  access. 

Almost  all  of  the  socio-demographic  characteristics  included  in  the  models  were 
significant  predictors  of  trying  to  find  information  on  new  benefits  and  changes,  but  far  fewer 
were  for  the  questions  on  managed  care  plans  and  supplemental  insurance.  Males  were  less 
likely  to  have  tried  to  find  information  about  new  benefits  or  changes  in  Medicare,  as  were 
beneficiaries  between  ages  65-75.  Respondents  over  age  75  were  less  likely  to  have  tried  to  find 
information  in  all  three  questions  on  access  to  information.  This  age  group  may  be  difficult  to 
reach  with  new  information  about  all  of  these  topics  and  may  require  special  efforts  to  present 
appropriate  information  to  them.  Non-whites  were  less  likely  to  have  tried  to  find  information 
for  only  the  question  about  managed  care  plans.  Progressively  higher  education  was  a 
significant  predictor  of  access  to  information  in  the  question  concerning  access  to  information  to 
information  concerning  new  benefits  and  changes  in  Medicare,  while  college-educated 
beneficiaries  were  more  likely  to  have  tried  to  find  information  about  managed  care  plans  than 
beneficiaries  with  an  8th  grade  education  or  less. 

Concerning  health  status  characteristics,  beneficiaries  in  fair  or  poor  health  were  more 
likely  to  have  tried  to  find  information  on  new  benefits  and  changes  in  Medicare  relative  to 
beneficiaries  in  excellent  or  very  good  health.  These  beneficiaries  may  be  interested  in  learning 
about  benefits  and  changes  that  can  help  them  with  current  illnesses  or  needs.  Surprisingly, 
health  status  was  not  a  significant  factor  for  the  questions  on  managed  care  plans  and 
supplemental  insurance. 
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Respondents  with  some  managed  care  enrollment  were  more  likely  to  have  tried  to  find 
information  concerning  the  availability  and  benefits  of  Medicare  managed  care  plans,  while 
beneficiaries  with  supplemental  insurance  were  less  likely  to  have  tried  to  find  this  information. 
Beneficiaries  with  supplemental  insurance  may  be  content  with  regular  Medicare  and  not 
interested  in  risking  entry  into  managed  care.  Respondents  with  Medicaid  eligibility  were  less 
likely  to  have  tried  to  find  information  about  new  benefits  or  changes  in  Medicare,  perhaps 
because  most  of  their  insurance  needs  are  met  through  a  combination  of  both  programs. 
Beneficiaries  with  higher  total  Medicare  charges  were  more  likely  to  have  tried  to  find 
information  concerning  new  benefits  and  changes  in  Medicare  and  concerning  managed  care 
plans  than  beneficiaries  without  any  Medicare  charges. 

Respondents  with  increasingly  higher  Perceived  Medicare  Knowledge  index  scores  were 
more  likely  to  have  tried  to  find  information  concerning  all  three  questions.  Respondents  who 
think  they  know  more  about  Medicare  appear  to  still  continue  to  look  for  information  concerning 
the  program  in  order  to  update  their  understanding  of  the  program.  In  contrast,  respondents  who 
had  never  heard  of  the  Internet  were  less  likely  to  look  for  information  on  new  benefits  and 
changes  in  Medicare  than  beneficiaries  who  did  not  have  access  to  the  Internet. 

In  summary,  reading  the  1999  Medicare  &  You  Handbook  either  thoroughly  or  in  part 
had  a  positive  but  small  effect  (ranging  from  one  to  three  per  cent)  on  MCBS  questions  used  to 
portray  access  to  information  and  was  significant  for  only  the  question  on  trying  to  find 
information  on  new  benefits  and  changes  in  Medicare.  The  NMEP  does  not  seem  to  have  much 
of  an  effect  on  access  to  information  as  measured  by  these  questions. 

4.2  Awareness 

We  analyzed  four  questions  that  reflect  the  NMEP  goal  of  Awareness.  These  questions 
and  the  policy  research  questions  they  address  are  listed  in  Exhibit  3.  Descriptive  frequencies 
for  each  of  the  MCBS  survey  questions  are  shown  in  bar  charts  in  the  Appendix  in  Exhibits  Cl- 
C4. 


Exhibit  3.  Awareness  Research  Questions  and  Related  MCBS  Questions 


II.  Awareness 

How  aware  are  beneficiaries  about  their  rights  and 
options  after  NMEP  implementation? 

What  types  of  Medicare  + 
Choice  health  plan  options  have 
beneficiaries  heard  of? 

BK64  -  Before  now,  had  you  ever  heard  of  a  Medical 
Savings  Account? 

BK66  -  Have  you  ever  heard  of  a  Preferred  Provider 
Organization,  or  PPO? 

Have  beneficiaries  heard  of  the 
buy-in  programs? 

BN8  -  Have  you  ever  heard  of  any  of  these  (Medicare  Buy- 
In)  programs? 

BN9  -  Were  you  aware  that  such  (Medicare  Buy-In) 
benefits  were  available? 

Two  of  these  questions  dealt  with  respondent's  awareness  of  Medicare  +  Choice 
options —  specifically,  medical  saving  accounts  (MSAs)and  preferred  provider  organizations 
(PPOs).  The  remaining  two  questions  concerned  awareness  of  the  buy-in  programs  which  are 
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open  to  seniors  and  persons  with  disabilities  who  have  limited  financial  resources  and  who  need 
help  paying  Medicare-related  costs.  Only  16.4%  of  respondents  were  aware  of  Medicare  MSAs, 
while  24.0%  of  respondents  reported  awareness  of  preferred  provider  organizations.  Medical 
savings  accounts  have  been  discussed  in  the  popular  press,  but  few  beneficiaries  have  been  faced 
with  a  decision  regarding  them.  On  the  other  hand,  respondents,  or  perhaps  their  family  or 
friends,  have  been  forced  to  learn  about  preferred  provider  arrangements  which  are  common 
outside  of  Medicare. 

Only  9.5%  of  respondents  had  heard  of  the  various  buy-in  programs.  The  buy-in 
programs  mentioned  in  the  question  in  the  MCBS  were  the  Qualified  Medicare  Beneficiary 
program,  the  Specified  Low-income  Medicare  Beneficiary  program,  and  the  Qualifying 
Individual  program.  Of  the  remaining  beneficiaries,  91 .7%  were  unaware  that  this  type  of  help 
was  available  to  pay  for  premiums,  deductibles  and  copayments.  A  large  number  of  respondents 
are  ineligible  for  these  programs  given  their  income  status  or  the  lack  of  a  disability.  Still,  more 
efforts  customized  to  reach  needy  beneficiaries  with  these  benefits  may  be  needed  in  order  to 
fulfill  their  intent. 

For  the  multivariate  analysis,  we  identified  two  questions  from  a  series  of  questions  that 
ask  the  respondent  if  he/she  has  heard  of  a  particular  type  of  health  care  arrangement  or  plan. 

The  two  questions  used  were  drawn  from  Round  23  of  the  MCBS  and  are  the  first  two 
question  in  the  exhibit  above: 

■  Have  you  ever  heard  of  a  MSA?  (BK  64) 
-    Have  you  ever  heard  of  a  PPO?  (BK66) 

We  performed  separate  logistic  regressions  using  responses  to  each  of  the  two  questions 
as  dependent  variables  to  analyze  predictive  factors  of  awareness.  Results  of  these  regressions 
are  presented  in  Exhibit  A3  in  Appendix  A.  Slightly  more  than  half  of  the  Medicare 
characteristics  modeled  had  a  significant  effect  on  responses  to  each  question,  most  at  the  p  < 
0.01  significance  level. 

As  a  measure  of  the  effect  of  the  NMEP,  we  calculated  the  marginal  effect  of  having  read 
the  Medicare  and  You  Handbook  in  the  5  states  that  received  it  for  each  of  the  two  questions 
modeled  above.  As  can  be  seen  in  Exhibit  4,  there  was  a  positive  significant  increase  in 
awareness  as  defined  by  these  questions  for  beneficiaries  who  read  the  handbook.  Beneficiaries 
who  read  the  handbook  in  the  five  states  were  about  8  percentage  points  more  aware  of  MSAs 
(about  22.1%  of  beneficiaries  who  read  the  handbook  had  heard  of  a  MSA  compared  to  only 
14.1%  of  beneficiaries  who  had  not  read  the  handbook).  Beneficiaries  who  read  the  handbook  in 
the  five  states  were  about  8.4  percentage  points  more  aware  of  PPOs  (about  29.8%  of 
beneficiaries  who  read  the  handbook  had  heard  of  a  PPO  compared  to  only  21.4%  of 
beneficiaries  who  had  not  read  the  handbook). 
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Exhibit  4.  Marginal  Effect  of  Reading  the  Handbook  in  the  Five  Intervention  States 


MCBS  Question 

Marginal  Effect 

Have  you  ever  heard  of  a  MSA? 

7.96  pet.  Points** 

Have  you  ever  heard  of  a  PPO? 

8.42  pet.  Points** 

**  Significant  at  p<0.01 


Based  on  the  results  in  Exhibit  A3,  almost  all  of  the  socio-demographic  characteristics 
included  in  the  models  were  significant  predictors  of  awareness  for  the  two  questions  on  MSAs 
and  PPOs.  Male  beneficiaries  were  more  likely  to  have  heard  of  both  a  MSA  and  a  PPO  than 
females.  Greater  male  awareness  may  be  due  in  part  to  males  traditionally  being  the  household 
member  who  primarily  attend  to  financial  affairs.  Beneficiaries  age  65  and  over  were  less  likely 
to  have  heard  of  a  PPO  than  beneficiaries  under  age  65.  Younger  beneficiaries  may  be  more 
familiar  with  managed  care,  and  thus  more  familiar  with  PPOs.  Beneficiaries  who  were  non- 
white  were  less  likely  to  have  heard  of  both  MSAs  and  PPOs.  Progressively  higher  education 
was  also  a  highly  significant  predictor  in  both  questions. 

Concerning  health  status  characteristics,  beneficiaries  in  fair  or  poor  health  were  less 
likely  than  beneficiaries  in  excellent  or  very  good  health  to  have  heard  of  both  MSAs  and  PPOs. 
Elderly  beneficiaries  who  are  very  ill  perhaps  have  little  need  for  PPOs  or  are  unable  to  take  the 
initiative  to  learn  about  these  options.  Beneficiaries  in  good  health  were  also  less  likely  than 
beneficiaries  in  excellent  or  very  good  health  to  have  heard  of  a  PPO.  Beneficiaries  who  make 
their  own  health  decisions  or  receive  help  making  health  decisions  were  more  likely  to  have 
heard  of  a  MSA  than  beneficiaries  who  had  someone  else  make  their  health  decisions  for  them. 
These  beneficiaries  would  naturally  have  more  awareness  of  their  health  plan  choices  than 
beneficiaries  who  have  health  decisions  made  for  them. 

Beneficiaries  who  had  any  managed  care  enrollment  were  more  likely  to  have  heard  of 
PPOs  than  beneficiaries  never  enrolled  in  a  managed  care  plan.  Managed  care  enrollment  may 
force  beneficiaries  to  be  more  aware  of  their  options.  Beneficiaries  enrolled  in  both  Medicare 
and  a  private  health  insurance  plan  were  more  likely  to  have  heard  of  both  MSAs  and  PPOs  than 
beneficiaries  enrolled  in  Medicare  only. 

Beneficiaries  with  higher  scores  on  the  Perceived  Medicare  Knowledge  index  were  more 
likely  to  know  about  MSAs  and  PPOs.  Both  ownership  of  a  personal  computer  and  access  to  the 
Internet  had  positive  effects  on  awareness  of  MSAs  and  PPOs.  Perhaps  beneficiaries  with 
personal  computers  and  Internet  access  take  more  initiative  to  be  aware  of  health  insurance 
issues.  Beneficiaries  who  had  never  heard  of  the  Internet  were  also  less  likely  to  have  heard  of 
PPOs  than  beneficiaries  with  Internet  access. 


s 


13 


4.3  Understanding 


We  analyzed  24  questions  reflecting  the  NMEP  goal  of  Understanding.  These  questions 
and  the  policy  research  questions  they  address  are  listed  in  Exhibit  5.  Questions  BK43-50 
comprise  the  8-item  quiz,  questions  BN1-5  comprise  the  Perceived  Medicare  Knowledge  index, 
and  questions  BN16,  18,  and  19  comprise  the  3-item  quiz — the  three  knowledge  indices  which 
will  be  analyzed  in  the  multivariate  analysis  in  this  section.  Descriptive  frequencies  for  each  of 
the  individual  MCBS  survey  questions  are  shown  in  bar  charts  in  the  Appendix  in  Exhibits  Dl- 
D24 

In  questions  BK1,  beneficiaries  were  asked  how  much  they  thought  they  knew  about  the 
Medicare  program.  Respondents  gave  one  of  the  five  answers  below: 

■  just  about  everything  I  need  to  know  (9.5%); 

■  most  of  what  I  need  to  know  (24.9%); 
•    some  of  what  I  need  to  know  (29.0%); 

■  little  of  what  I  need  to  know  (21.1%);  and 
almost  none  of  what  I  need  to  know  (1 5.5%). 

There  is  a  wide  distribution  of  responses  to  this  question.  While  this  question  asks  respondents 
to  describe  their  general  knowledge  of  the  Medicare  program  as  a  whole,  it  may  not  be  that 
useful  because  it  measures  self-reported  knowledge  and  may  not  be  specific  enough  to  gain 
much  insight  into  what  respondents  say  they  know  and  don't  know. 

Four  additional  questions  provide  information  on  beneficiaries'  self-reported 
understanding  of  selected  Medicare  +  Choice  plan  options.  Respondents  were  asked  how  well 
they  understood  MSAs  (BK65),  PPOs  (BK67),  PSOs  (BK70),  and  POSs  (BK73).  They  could 
respond  with  one  of  four  choices — Very  well,  Fairly  well,  Not  too  well,  or  Not  at  all. 
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Exhibit  5.  Understanding  Research  Questions  and  Related  MCBS  Questions 


III.  Understanding 

How  knowledgeable  are  beneficiaries  about  Medicare 
after  NMEP  implementation? 

What  is  beneficiaries'  self-reported 
understanding  of  the  Medicare  program 
in  general? 

BK1-  How  much  do  you  think  you  know  about  the  Medicare  program? 

What  is  beneficiaries'  self-reported 
understanding  of  selected  Medicare  + 
Choice  plan  options? 

BK65  -  How  well  do  you  understand  MSAs:  Would  you  say  very  well, 
fairly  well,  not  too  well,  or  not  at  all? 

BK67  -  How  well  do  you  understand  PPOs:  Would  you  say  very  well, 
fairly  well,  not  too  well,  or  not  at  all? 

BK70  -  How  well  do  you  understand  PSOs:  Would  you  say  very  well, 
fairly  well,  not  too  well,  or  not  at  all? 

BK73  -  How  well  do  you  understand  POSs:  Would  you  say  very  well, 
fairly  well,  not  too  well,  or  not  at  all? 

How  informed  are  beneficiaries  about 
what  plan  options  are  available  to 
them? 

BK68  -  Can  people  on  Medicare  join  PPOs? 
BK71  -  Can  people  on  Medicare  join  PSOs? 
BK74  -  Can  people  on  Medicare  join  POSs? 

How  informed  are  beneficiaries  about 
the  Medicare  program  and  selected 
plan  options?  (8-item  quiz) 

Series  of  true/false  questions: 

BK43  -  Most  people  covered  by  Medicare  can  select  among  different 
kinds  of  health  plan  options  within  Medicare? 

BK44  -  Medicare  without  a  supplemental  insurance  policy  pays  for  all 
of  your  health  care  expenses? 

BK45  -  If  you  are  happy  with  the  way  you  currently  receive  health  care, 
you  do  not  have  to  make  any  changes  in  the  way  you  get  your  Medicare 
services. 

BK46  -  The  Medicare  program  has  recently  begun  to  offer  more 
information  and  help  in  order  to  answer  your  Medicare  questions. 
BK47  -  People  can  report  complaints  to  Medicare  about  their  Medicare 
managed  care  plans  (HMOs)  or  supplemental  plans  if  they  are  not 
satisfied  with  them. 

BK48  -  If  someone  joins  a  Medicare  managed  care  plan  (HMO)  that 
covers  people  on  Medicare,  they  have  limited  choices  about  which 
doctors  they  can  see. 

BK49  -  If  someone  joins  a  Medicare  managed  care  plan  (HMO)  that 
covers  people  on  Medicare,  they  can  change  or  drop  the  plan  and  still  be 
covered  by  Medicare. 

BK50  -  Medicare  managed  care  plans  (HMOs)  that  cover  people  on 
Medicare  often  cover  more  health  services,  like  prescribed  medicines, 
than  Medicare  without  a  supplemental  policy. 

How  much  do  beneficiaries  feel  they 
know  about  various  Medicare  issues? 
(Perceived  Medicare  Knowledge  index) 

BN1  -  How  much  do  you  feel  you  know  about  what  medical  services 
Medicare  covers  or  does  not  cover? 

BN2  -  How  much  do  you  feel  you  know  about  how  much  you  have  to 
pay  for  medical  services? 

BN3  -  How  much  do  you  feel  you  know  about  supplemental  or 
Medigap  insurance,  such  as  what  it  covers  or  how  it  works  with 
Medicare  to  pay  medical  claims? 

BN4  -  How  much  do  you  feel  you  know  about  the  availability  and 
benefits  of  Medicare  managed  care  plans? 

BN5  -  How  much  do  you  feel  you  know  about  choosing  or  finding  a 
doctor  or  other  health  care  provider? 

How  well  do  beneficiaries  understand 
the  Medicare  program?  (3-item  quiz) 

Series  of  true/false  questions: 

BN16  -Medicare  covers  colorectal  cancer  screening. 

BN 1 8  -  Medigap  or  supplemental  insurance  is  the  same  as  a  Medicare 

managed  care  plan. 

BN19  -  Medicare  covers  an  annual  flu  shot. 
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About  39.6%  of  respondents  indicated  that  they  understood  MSAs  very  well  or  fairly 
well.  In  other  words,  the  majority  of  respondents  had  a  limited  understanding  of  MSAs. 
Responses  to  the  questions  on  understanding  of  PPOs,  PSOs,  and  POSs  show  a  decreasing 
understanding  of  each  of  these  arrangements  given  the  order  of  their  listing.  While  almost  49% 
of  respondents  said  they  understood  PPOs  very  well  or  fairly  well,  only  about  40%  of 
respondents  said  the  same  for  PSOs,  and  only  about  36%  said  the  same  for  POSs.  PPOs  have 
been  in  the  marketplace  longer  than  PSOs  and  POSs  and  respondents  may  be  more  generally 
familiar  with  PPOs  than  the  other  arrangements.  On  the  other  hand,  beneficiaries  may  not 
realize  that  a  PPO  is  a  specific  model  and  think  it  refers  to  any  form  of  network  plan.  Some 
insurers  call  HMOs  and  POSs  preferred  provider  organizations. 

Questions  BK68,  71,  and  74  ask  respondents  about  plan  options  available  to  them  in 
Medicare.  About  56%  of  respondents  said  that  they  could  join  PPOs,  while  about  59%  said  they 
could  join  PSOs  and  62%  said  they  could  join  POSs.  When  comparing  these  responses  to  those 
in  the  prior  paragraph  about  how  well  respondents  understood  each  of  these  options, 
arrangements  beneficiaries  said  they  could  more  frequently  join  were  arrangements  they 
understood  less  well.  POSs  were  less  understood,  but  more  beneficiaries  replied  that  they  could 
join  them.  This  finding  may  be  evidence  of  social  desirability  bias. 

Correct  responses  on  individual  questions  of  the  8-item  quiz  ranged  from  39%  of 
respondents  who  answered  BK50  correctly  as  True  (that  managed  care  plans  cover  more  services 
than  Medicare  without  a  supplemental  policy)  to  77%  of  respondents  who  answered  BK44 
correctly  as  False  (that  Medicare  without  a  supplemental  insurance  policy  pays  for  all  of  your 
health  care  expenses).  All  of  these  questions  but  one  had  True  as  the  correct  answer. 
Respondents  may  have  been  more  challenged  if  more  answers  had  been  False. 

In  the  Perceived  Medicare  Knowledge,  respondents  reported  how  much  they  felt  they 
knew  about  particular  aspects  of  the  Medicare  program.  When  these  responses  are  aggregated 
into  the  knowledge  index,  they  are  a  valid  and  internally  consistent  measure  of  how  much 
respondents  feel  they  know  about  particular  aspects  of  Medicare. 

Correct  responses  on  individual  questions  in  the  3-item  quiz  ranged  from  44%  of 
respondents  who  answered  BN18  correctly  as  False  (that  Medigap  or  supplemental  insurance  is 
the  same  as  a  Medicare  managed  care  plan)  to  84%  of  respondents  who  answered  BN19 
correctly  as  True  (that  Medicare  covers  an  annual  flu  shot).  The  third  question  on  whether 
Medicare  covers  colorectal  cancer  screening  was  answered  correctly  by  49%  of  respondents. 

Weighted  mean  scores  for  the  three  knowledge  indices  are  shown  in  Exhibit  6. 


Exhibit  6.  Weighted  Mean  Knowledge  Index  Scores 


MCBS  Knowledge  Index 

Weighted  Mean 

Standard  Error 

8-item  quiz 

59.0% 

14.5% 

Perceived  Medicare  Knowledge 

56.2 

13.3 

3-item  quiz 

51.0 

25.0 
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Each  of  these  three  knowledge  indices  were  analyzed  bivariately  with  a  set  of  eight 
beneficiary  characteristics  to  understand  how  scores  on  each  of  the  knowledge  indices  varied 
within  each  characteristic.  Beneficiary  characteristics  chosen  for  this  bivariate  analysis  were 
gender,  age  group,  race,  education,  self-reported  health  status,  whether  the  respondent  had  any 
managed  care  enrollment,  type  of  insurance,  and  the  amount  of  Medicare  covered  charges. 
Knowledge  scores  were  estimated  for  the  entire  beneficiary  population  and  reported  with 
confidence  intervals.  These  results  are  presented  in  Exhibit  7. 

Across  the  three  knowledge  indices,  results  from  the  bivariate  analyses  were  consistent 
among  beneficiary  characteristics.  There  was  very  little  difference  in  knowledge  scores 
concerning  gender.  Beneficiaries  between  ages  65-75  scored  the  highest  on  each  index,  while 
beneficiaries  under  65  years  of  age  scored  lowest,  perhaps  because  they  are  newest  to  the 
Medicare  program.  There  were  wide  differences  in  knowledge  score  concerning  race,  where 
white  beneficiaries  scored  higher  than  non-white  beneficiaries.  As  expected,  respondents  with 
progressively  higher  education  had  higher  knowledge  scores. 

Respondents  with  progressively  better  health  had  higher  knowledge  scores.  Respondents 
with  some  managed  care  enrollment  had  higher  knowledge  scores  than  beneficiaries  without 
managed  care  enrollment.  This  was  expected  as  some  of  the  questions  concerned  managed  care. 
Respondents  with  Medicare  and  supplemental  insurance  had  the  highest  knowledge  scores  while 
person  on  Medicaid  had  the  lowest  knowledge  scores.  While  results  for  the  amount  of  Medicare 
covered  charges  were  not  monotonic  within  the  category  or  between  knowledge  indices,  overall 
there  was  very  little  variance  in  knowledge  scores  for  this  characteristic. 

We  performed  OLS  regressions  on  each  knowledge  index  expressed  as  a  scale  from  0  to 
100.  These  results  are  presented  in  Exhibit  A4  in  Appendix  A.  The  knowledge  score  can  thus 
be  interpreted  as  a  percentage  score  out  of  1 00  points. 

As  a  measure  of  the  effect  of  the  NMEP,  we  calculated  the  marginal  effect  of  having  read 
the  Medicare  and  You  handbook  in  the  5  states  that  received  it  for  each  of  the  three  knowledge 
indices  modeled  above.  As  can  be  seen  in  Exhibit  8,  there  was  a  positive  significant  increase  in 
knowledge  for  all  three  knowledge  indices  for  beneficiaries  who  read  the  handbook. 
Beneficiaries  who  had  read  the  handbook  in  the  five  states  scored  7.3  percentage  points  higher 
on  the  8-item  quiz  than  beneficiaries  who  did  not  read  the  handbook.  Beneficiaries  who  had  read 
the  handbook  in  the  five  states  scored  5.9  percentage  points  higher  on  the  Perceived  Medicare 
Knowledge  index  than  beneficiaries  who  did  not  read  the  handbook.  Beneficiaries  who  had  read 
the  handbook  in  the  five  states  scored  4.8  percentage  points  higher  on  the  3 -item  quiz  than 
beneficiaries  who  did  not  read  the  handbook.  All  of  these  results  are  highly  significant. 
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Exhibit  7 

Mean  Knowledge  Index  Scores  (in  Percent)  for  Selected  Sub  Groups 

(n=30, 105,201) 

Perceived 

8-item  quiz         Medicare  Knowledge         3-item  quiz 


Sub  Group 

Mean 

95%  Conf.  Int. 

Mean 

95%  Conf.  Int. 

Mean 

95%  Conf.  Int. 

Gender 

Male 
Female 

j  /  .0 

55.1 

(J J.O,  J7.0 ) 

(53.3,  56.9) 

O  7 
JZ.Z 

51.1 

/en  8  ^i  (\\ 
(50.7,  52.5) 

^8  7 

JO.  / 

59.2 

OKI. J ) 

(57.6, 60.8) 

Age  Group 

Under  65 
65-75 
Over  75 

48.8 
jy.  1 
54.0 

(46.6,51.0) 

[?  I.I,  Ol .  1) 

(52.2,  55.8) 

43.5 
sa  i 

j4. 1 

51.0 

(41.7, 45.3) 
pz.y,  j  j.  j ) 
(49.4,  52.6) 

48.3 

£7  5 
OZ.  J 

57.9 

(45.9,  50.7) 

On.  1  j 

(56.3,  59.5) 

Race 

White 
Non-white 

58.8 

A  A  C 

44. J 

(56.8,  60.8) 

1 A  1  Q   A 7  1  \ 

(4i.y,  4 /.i ) 

54.3 

1Q  TP 

(52.9,  55.7) 

(17  ^   A  1  Q\ 

(j/.j,  41. y) 

62.1 

A<\  1 
4j.1 

(60.3,  63.9) 

{4j.j,  'tO.y; 

Education 

Less  than  8th  grade 
Some/all  high  school 
Some/all  college 

43.9 

JOJ 

64.1 

(41.3,46.5) 

(j4.j,  Jo.  j ) 

(62.7,  65.5) 

40.7 
jZ.U 
58.3 

(38.7,  42.7) 

^jU.o,  jj.ZJ 

(56.9,  59.7) 

48.0 

^8  Q 
Jo. 7 

66.5 

(45.6,  60.4) 

^J  /.J,  OU. J _) 

(64.9,  68.1) 

Health  status 

Excellent/V ery  Good 

Good 

Fair/Poor 

61.0 
ee  c 

J  J.O 

49.7 

(59.2,  62.8) 

[pj.O,  jo.V) 

(47.5,51.9) 

55.5 
e  i  < 

45.5 

(54.3,  56.7) 

(AQ  7  ^1 
(Hy.  /,  J  J.  J  ) 

(43.9, 47.1) 

61.9 

J7.J 

54.1 

(60.5,  63.3) 

\  J  /  .  J,  Ol.Jj 

(51.9,  56.3) 

Managed  Care  Enrollment 

Some 
None 

68.5 
54.3 

(66.7,  70.3) 
(52.3,  56.3) 

57.2 
50.7 

(55.4,  59.0) 
(49.3,  52.1) 

61.2 
58.6 

(59.4,  63.0) 
(57.0,  60.2) 

Type  of  Health  Insurance 

Medicare  only 
Medicare  and  private 
Medicare  and  Medicaid 

56.6 
59.4 
43.4 

(54.2,  59.0) 
(57.6,61.2) 
(41.2, 45.6) 

48.0 
55.7 
40.3 

(46.2,  49.8) 
(54.3,57.1) 
(38.5,  42.1) 

52.8 
63.9 
48.2 

(51.2,  54.4) 
(62.1,65.7) 
(46.0,  50.4) 

Health  Service  Utilization 

None 
$l-$499 
$500-$4,999 
$5,000  and  over 

58.1 
55.3 
57.0 
54.1 

(55.9,  60.3) 
(52.9,  57.7) 
(55.0,  59.0) 
(52.1,56.1) 

50.1 
50.5 
53.0 
51.6 

(48.3,51.9) 
(48.7,  52.3) 
(51.4,  54.6) 
(50.0,  53.2) 

53.0 
57.6 
62.0 
60.9 

(51.4,  54.6) 
(55.4,  59.8) 
(60.2, 63.8) 
(59.3,  62.5) 
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Exhibit  8.  Marginal  Effect  of  Reading  the  Handbook  in  the  Five  Intervention  States 


MCBS  Knowledge  Index 

Marginal  effect 

Std.  Err. 

8-item  quiz 

7.33**  percentage  points 

0.93 

Perceived  Medicare  Knowledge  index 

5.92**  percentage  points 

0.98 

3 -item  quiz 

4.76**  percentage  points 

1.21 

**  Significant  at  p<0.01 


These  modest  gains  in  knowledge  (from  roughly  5-7  percentage  points)  are  consistent 
with  increases  in  knowledge  from  other  interventions  described  in  the  research  literature.  There 
may  be  additional  increases  in  knowledge  over  time  with  additional  mailings  of  the  handbook. 
The  highest  increases  in  knowledge  for  having  read  the  handbook  were  shown  in  the  8-item  quiz 
and  the  Perceived  Medicare  Knowledge  index,  both  of  which  have  been  shown  to  be  internally 
consistent  and  valid  measures  of  knowledge  of  Medicare  beneficiaries  in  RTFs  Technical  Report 
1:  Knowledge  Index. 

There  was  little  difference  in  knowledge  scores  according  to  gender  (even  though  the 
regression  results  show  a  significant  difference  for  a  1 .4%  increase  in  knowledge  for  males  on 
the  8-item  quiz).  There  was  approximately  a  3  percentage  point  increase  for  beneficiaries 
between  ages  65-75  as  compared  to  beneficiaries  less  than  age  65  in  the  8-item  quiz  and  the 
Perceived  Medicare  Knowledge  index.  In  the  3-item  quiz,  beneficiaries  between  ages  65-75 
scored  7  percentage  points  higher  and  person  over  age  75  scored  3  percentage  points  higher  than 
beneficiaries  who  were  less  than  age  65.  Beneficiaries  who  were  non-white  scored  between  5.7 
and  8.9  percentage  points  lower  than  beneficiaries  who  were  white,  independent  of  educational 
status. 

Respondents  with  some  high  school  or  a  high  school  degree  scored  roughly  4  to  6 
percentage  points  higher  than  respondents  with  an  eighth  grade  education  or  less,  and  college- 
educated  respondents  scored  roughly  9-10  percentage  points  higher  than  beneficiaries  with  an 
eighth  grade  education  or  less.  Respondents  in  both  the  8-item  quiz  and  the  Perceived  Medicare 
Knowledge  index  reporting  only  good  health  scored  about  2  percentage  points  lower  than 
respondents  reporting  excellent  or  very  good  health,  while  respondents  with  fair  or  poor  health  in 
quizzes  scored  about  4.5  percentage  points  lower  than  respondents  reporting  excellent  or  very 
good  health. 

Although  there  were  no  differences  in  knowledge  scores  for  proxy  respondents  when 
compared  to  those  of  beneficiaries  who  responded  for  themselves,  there  were  marked  differences 
for  respondents  who  made  their  own  health  care  decisions  or  had  help  making  them  as  opposed 
to  those  who  had  someone  else  make  their  health  care  decisions  for  them.  Respondents  who 
made  their  own  decisions  scored  from  1 0  to  13  percentage  points  higher  than  those  who  had 
someone  else  make  their  decisions  in  all  three  quizzes.  Respondents  who  had  help  in  their 
decisions  scored  between  6  to  1 0  percentage  points  higher  than  beneficiaries  who  had  someone 
else  make  their  decisions  for  them  in  all  three  quizzes.  Beneficiaries  have  higher  knowledge 
scores  to  the  degree  that  they  can  take  responsibility  for  their  own  decisions,  partly  because  they 
may  be  required  to  use  some  of  their  knowledge  to  make  health  insurance  decisions. 
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Beneficiaries  with  managed  care  enrollment  experience  scored  12  to  18  percentage  points 
higher  than  beneficiaries  without  managed  care  experience.  These  beneficiaries  are  required  to 
review  materials  and  make  decisions  regarding  their  managed  care  arrangements  on  a  more 
frequent  basis  than  beneficiaries  without  managed  care  given  the  many  changes  in  the  Medicare 
program  concerning  managed  care. 

Respondents  with  supplemental  insurance  scored  approximately  4  to  7  percentage  points 
higher  than  beneficiaries  with  Medicare  only  in  all  three  quizzes,  while  beneficiaries  with 
Medicaid  scored  3  percentage  points  lower  than  beneficiaries  with  Medicare  only  on  the  8-item 
quiz.  Respondents  who  make  decisions  regarding  supplemental  insurance  issues  require  more 
knowledge  to  make  decisions  on  health  insurance,  but  perhaps  not  quite  as  much  as  beneficiaries 
who  are  in  managed  care  arrangements. 

Respondents  with  progressively  higher  Medicare  covered  charges  scored  higher  than 
beneficiaries  with  no  expenditures.  Beneficiaries  with  even  only  $1  to  $499  in  charges  scored  4 
to  7  percentage  points  higher  than  beneficiaries  without  charges,  and  beneficiaries  with 
expenditures  greater  than  $500  scored  between  6  to  12  percentage  points  higher.  Apparently  use 
of  health  care  resources  promotes  better  understanding  of  the  resources  at  hand. 

Beneficiaries  with  cable  TV,  personal  computers,  or  Internet  access  scored  roughly  2  to  3 
percentage  points  higher  than  beneficiaries  without  these  resources,  and  beneficiaries  who  had 
never  heard  of  the  Internet  scored  about  1  percentage  point  lower  than  beneficiaries  who  did  not 
have  Internet  access. 

4.4  Impact/Use 

We  analyzed  five  questions  that  reflect  the  NMEP  goal  of  Impact/Use.  These  questions 
and  the  policy  research  questions  they  address  are  listed  in  Exhibit  9.  Descriptive  frequencies 
for  each  of  the  MCBS  survey  questions  are  shown  in  bar  charts  in  the  Appendix  in  Exhibits  El- 
E5. 


Exhibit  9.  Impact/Use  Research  Questions  and  Related  MCBS  Questions 


IV.  Impact 

What  is  the  effect  of  the  NMEP  on  selected  beneficiary 
activities  and  outcomes? 

How  did  beneficiaries  use  the  1 999 
Medicare  &  You  bulletin  and  handbook? 
What  actions  did  they  take  as  a  result  of 
being  exposed  to  them? 

BK30  -  Would  you  say  you  have  read  this  (book/bulletin)  thoroughly, 
that  you  have  read  parts  of  it,  or  that  you  haven't  read  it  at  all? 
BK32  -  Have  you  ever  used  this  (book/bulletin)  to  find  information 
about  health  plan  options  available  to  you,  such  as  Medicare  managed 
care  plans,  HMOs  or  supplemental  plans? 

What  was  beneficiaries'  information 
seeking  behavior  after  the 
implementation  of  the  NMEP? 

BK58  -  Starting  in  1999,  Medicare  will  offer  new  health  plan  options. 
You  may  want  to  review  these  options.  Have  you  reviewed  information 
about  different  Medicare  health  plan  options? 

How  satisfied  are  beneficiaries  with  the 
information  they  receive  from  and  can 
give  to  the  Medicare  program? 

BK2  -  How  satisfied  are  you  in  general  with  the  availability  of 
information  about  the  Medicare  program  when  you  need  it? 

How  confident  are  beneficiaries  in  their 
ability  to  select  an  appropriate  plan? 

BK62  -  How  confident  are  you  that  you  could  choose  a  health  care  plan 
that  best  matches  your  needs?  Would  you  say  not  confident,  slightly 
confident,  moderately  confident,  or  extremely  confident? 

20 


Respondents  were  asked  in  Question  BK30  if  they  had  read  the  handbook  or  the  bulletin 
thoroughly,  read  parts  of  it,  or  read  none  at  all.  Respondents  who  indicated  that  they  had  not 
received  the  handbook  or  bulletin  after  being  shown  a  copy  of  it  in  Question  BK28  were  coded 
as  if  they  did  not  read  the  handbook  or  bulletin  at  all  for  Question  BK30.  Using  this  coding, 
26.4%  of  all  respondents  indicated  that  they  read  the  book  or  bulletin  thoroughly  or  read  parts  of 
it.  This  variable  is  used  as  the  key  policy  intervention  variable  in  the  multivariate  analyses. 

Respondents  were  then  asked  in  Question  BK32  if  they  had  used  the  handbook  or  bulletin 
to  find  information  about  health  plan  options  available  to  them,  such  as  Medicare  managed  care 
plans,  HMOs,  or  supplemental  plans.  Only  8.4%  of  respondents  indicated  that  they  used  the 
handbook  or  bulletin  to  do  so.  Given  the  relatively  small  percentage  of  Medicare  beneficiaries 
involved  in  managed  care,  a  lower  percentage  was  expected  for  this  question. 

In  Question  BK58,  respondents  were  asked  if  they  had  reviewed  information  about 
different  Medicare  health  plan  options.  Only  12.3%  of  respondents  indicated  that  they  had  done 
so,  which  corresponds  to  the  low  percentage  of  beneficiaries  who  used  the  handbook  or  bulletin 
in  Question  BK32  just  discussed. 

One  question  addressed  respondent  satisfaction.  In  Question  BK2,  93%  of  respondents 
reported  that  generally  they  were  either  very  satisfied  or  satisfied  with  the  availability  of 
information  about  the  Medicare  program  when  they  needed  it.  This  finding  may  mean  that 
beneficiaries  are  happy  with  their  insurance  in  general.  On  the  other  hand,  given  the  changes  in 
the  Medicare  program  in  recent  years,  this  may  be  a  reflection  of  how  much  beneficiaries  don't 
know,  and  really  should.  Additional  follow-up  questions  asked  in  the  next  section  about  the 
HCFA-sponsored  materials  may  be  informative  for  qualifying  responses  to  this  question. 

In  Question  BK62,  respondents  were  asked  how  confident  they  were  that  they  could 
choose  a  health  care  plan  that  best  matches  their  need.  About  1 1 .2%  of  respondents  said  they 
were  extremely  confident,  18.6%  said  they  were  very  confident,  34.4%  of  respondents  said  they 
were  moderately  confident,  29.2%  said  they  were  slightly  confident,  and  6.6%  said  they  were  not 
confident  in  making  this  choice.  Almost  two-thirds  of  respondents  indicated  they  were  at  least 
moderately  confident  in  making  this  choice. 

The  impact,  use,  and  usefulness  of  the  NMEP  was  explored  through  the  multivariate 
analysis  of  four  survey  questions.  They  are: 

■  Have  you  ever  used  this  book/bulletin  to  find  information  about  health  plan  options 
available  to  you/spouse,  such  as  Medicare  managed  care  plans,  HMOs  or 
supplemental  plans?  (BK32) 

«  Starting  in  1 999,  Medicare  will  offer  new  health  plan  options.  You  may  want  to 
review  these  options.  Have  you  reviewed  information  about  different  Medicare 
health  plan  options?  (BK58) 

•    How  satisfied  are  you  in  general  with  the  availability  of  information  about  the 
Medicare  program  when  you  need  it?  (BK2) 

■  How  confident  are  you  that  you  could  choose  a  health  care  plan  that  best  matches 
your  needs?  (BK62) 
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Questions  BK32  and  BK58  were  analyzed  using  a  logistic  regression  model  with  a 
bivariate  outcome  (yes,  no),  and  questions  BK2  and  BK62  were  analyzed  using  an  ordered 
logistic  regression  model.  Results  of  these  four  models  are  presented  in  Exhibit  A5  in  Appendix 
A. 

As  can  been  seen  in  Exhibit  10,  the  models  predict  that  those  who  read  the  handbook 
were  15.6  percentage  points  more  likely  to  say  that  they  actually  used  the  handbook  to  find  out 
about  available  health  plan  options  in  question  BK32  (16.7%  of  those  who  read  the  handbook 
relative  to  1 .2%  of  those  who  did  not).  Also,  in  response  to  question  BK58,  those  who  read  the 
handbook  were  12.2  percentage  points  more  likely  to  have  reviewed  information  (using  any 
source)  about  different  health  plan  options  (20.7%  of  those  who  read  the  handbook  relative  to 
8.5%  of  those  who  did  not).  In  contrast,  no  significant  difference  was  found  in  the  level  of 
satisfaction  or  confidence  in  making  a  health  plan  choice  between  those  in  the  five  states  who 
received  the  1999  Medicare  &  You  Handbook  and  those  who  did  not.  Based  on  these  results,  we 
see  that  while  those  who  received  the  handbook  were  significantly  more  likely  to  use  available 
information  to  make  decisions,  they  were  not  more  satisfied  with  the  availability  of  information 
or  their  confidence  in  choosing  a  health  plan. 


Exhibit  10.  Marginal  Effect  of  Reading  the  Handbook  in  the  Five  Intervention  States 


MCBS  Question 

Marginal  Effect 

Have  you  ever  used  this  book/bulletin  to  find 
information  about  health  plan  options  available  to 
you/spouse,  such  as  Medicare  managed  care  plans, 
HMOs  or  supplemental  plans? 

15.6  percentage  points  ** 

Starting  in  1 999,  Medicare  will  offer  new  health 
plan  options.  You  may  want  to  review  these 
options.  Have  you  reviewed  information  about 
different  Medicare  health  plan  options? 

12.2  percentage  points  ** 

How  satisfied  are  you  in  general  with  the 
availability  of  information  about  the  Medicare 
program  when  you  need  it? 

Not  significant 

How  confident  are  you  that  you  could  choose  a 
health  care  plan  that  best  matches  your  needs? 

Not  significant 

**  Significant  at  p<0.01 


Reading  the  handbook  in  the  five  states  was  positively  related  to  having  reviewed 
information  in  general  concerning  new  health  plan  options  and  in  having  used  the  handbook  to 
do  so.  However,  in  Questions  BK2  and  62,  respondents  were  significantly  less  satisfied  with  the 
availability  of  information  about  Medicare  when  they  needed  it  and  less  confident  that  they  could 
choose  a  health  plan  that  best  met  their  needs. 

Selected  demographic  characteristics  were  significant  predictors  of  whether  beneficiaries 
used  the  book/bulletin  to  review  information  on  their  health  plan  options  and  whether  they 
reviewed  information  from  any  source  on  their  new  health  plan  options  starting  in  1 999. 
Individuals  who  were  65  years  of  age  and  older  were  less  likely  to  have  reviewed  information  in 
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general  concerning  new  health  plans  but  no  differences  by  age  category  were  found  in  relation  to 
use  of  the  book/bulletin.  Beneficiaries  with  at  least  some  high  school  education  relative  to  those 
with  less  than  an  8th  grade  education  were  more  likely  to  have  used  the  handbook  to  find  out 
about  their  options,  as  were  those  with  some  college  education  relative  to  those  with  less  than  an 
8th  grade  education. 

Beneficiaries  with  Medicare  expenditures  of  any  amount  were  more  likely  to  have 
reviewed  information  about  their  health  plan  options  (BK  58)  as  were  those  with  some  managed 
care  enrollment.  Proxy  respondents  were  less  likely  to  have  used  the  book/bulletin  to  find  out 
information. 

Individuals  with  greater  Perceived  Medicare  Knowledge  index  scores  were  more  likely  to 
have  reviewed  information  on  new  health  plan  choices.    Those  who  had  never  heard  of  the 
Internet  were  less  likely  to  have  used  information  in  general  or  the  handbook  specifically. 

Males  were  less  likely  to  be  satisfied  with  the  availability  of  information  (BK  2)  as  were 
those  with  at  least  some  high  school  education.  Males  and  those  65  years  of  age  and  older  were 
less  likely  to  be  confident  that  they  could  choose  a  health  plan  to  meet  their  needs  (BK  62). 

Health  status  was  found  to  be  inversely  related  to  satisfaction  and  confidence. 
Individuals  who  reported  being  in  good  health  or  fair  to  poor  health  relative  to  excellent  or  very 
good  health  were  more  satisfied  with  the  availability  of  information  and  their  ability  to  choose  a 
health  plan  that  met  their  needs. 

Beneficiaries  with  Medicare  and  private  coverage,  relative  to  those  with  only  Medicare 
coverage,  were  significantly  less  confident  that  they  could  choose  a  health  plan  that  meets  their 
needs.  The  same  relationship  was  found  for  those  with  some  managed  care  enrollment  and 
those  who  make  their  own  health  insurance  decisions  (either  with  some  assistance  or  not)  relative 
to  those  who  do  not  make  their  own  decisions. 

Lastly,  those  who  scored  higher  on  the  Perceived  Medicare  Knowledge  index  were 
significantly  less  likely  to  be  satisfied  with  the  availability  of  information  and  less  confident  that 
they  could  choose  a  health  plan  that  met  their  needs. 

In  conclusion,  respondents  using  the  handbook  were  more  likely  to  use  information 
(including  the  handbook)  to  review  health  plan  options.  However,  beneficiaries  tended  to  be 
significantly  less  satisfied  with  the  information  they  had  about  Medicare  and  less  confident  with 
their  ability  to  make  health  plan  decisions  that  meet  their  needs.  These  results  may  be 
interpreted  as  meaning  that  those  who  have  used  the  greatest  amount  of  information  and  have  the 
greatest  perceived  knowledge  of  the  Medicare  program  have  a  greater  appreciation  of  how 
complicated  and  difficult  their  health  care  choices  are.  These  results  may  also  support 
continuing  efforts  to  improve  the  information  provided  to  beneficiaries  in  terms  of  its  usefulness 
and  usability. 
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4.5 


Direct  Evaluation  of  NMEP  Materials 


We  analyzed  four  questions  that  ask  beneficiaries  to  evaluate  the  NMEP  materials.  These 
questions  and  the  policy  research  questions  they  address  are  listed  in  Exhibit  11.  Descriptive 
frequencies  for  each  of  the  MCBS  survey  questions  are  shown  in  bar  charts  in  the  Appendix  in 
Exhibits  F1-F4. 


Exhibit  11.  NMEP  Materials  Policy  Questions  and  MCBS  Questions 


V.  Direct  Evaluation  of  NMEP 
Materials 

What  proportion  of  beneficiaries 
nationwide  received  the  1999  Medicare 
&  You  materials,  and  what  proportion 
kept  them?  What  were  beneficiaries' 
impressions  of  the  materials,  including 
the  comparison  charts,  in  terms  of 
comprehension? 

BK28  -  Did  you  receive  a  copy  of  this  (book/bulletin),  called  Medicare 
&  You,  in  the  mail  sometime  in  the  past  year? 

BK33  -  How  easy  to  understand  did  you  find  (this  book/this  bulletin/the 
parts  you  looked  at):  Would  you  say  (it  was/they  were)  very  easy  to 
understand,  somewhat  easy  to  understand,  somewhat  difficult  to 
understand,  or  very  difficult  to  understand? 

BK37  -  How  easy  did  you  find  (the  comparison  chart)  to  understand: 
Would  you  say  (it  was/they  were)  very  easy  to  understand,  somewhat 
easy  to  understand,  somewhat  difficult  to  understand,  or  very  difficult  to 
understand? 

BK40  -  Do  you  still  have  the  Medicare  &  You  (book/bulletin)? 

Only  39.4%  of  respondents  indicated  that  they  had  received  a  copy  of  the  handbook  or 
bulletin  in  the  past  year  in  Question  BK28.  Since  these  materials  were  mailed  to  all 
beneficiaries,  and  interviewers  showed  beneficiaries  a  copy  of  the  handbook  or  bulletin  to 
improve  recall,  it  is  troubling  that  such  a  low  percentage  said  they  received  it.  Beneficiaries  may 
not  remember  it  because  they  file  the  materials  away  and  use  them  only  when  they  have  a  need 
for  them.  They  may  also  not  have  been  the  person  who  picked  up  mail  to  their  house  that  day. 

In  Question  BK33,  respondents  rated  the  handbook  or  bulletin  in  terms  of  their  ease  in 
understanding  the  parts  they  looked  at.  About  30.5%  of  respondents  said  the  materials  were  very 
easy  to  understand  and  54.6%  said  they  were  somewhat  easy.  Only  2.4%  of  respondents  said  the 
materials  were  very  difficult  to  understand,  and  12.5%  said  they  were  somewhat  difficult  to 
understand.  When  asked  how  easy  they  found  the  comparison  chart  in  the  materials  to 
understand,  respondents  said  36.3%  of  respondents  said  it  was  very  easy  to  understand,  and 
53.5%  said  it  was  somewhat  easy  to  understand.  Only  about  12.3%  said  the  chart  was  somewhat 
difficult  or  very  difficult  to  understand.  Using  prior  analyses  from  the  Phase  1  report  RTI 
prepared  for  HCFA  in  July  2000,  we  know  that  63%  of  beneficiaries  who  received  the  handbook 
in  the  five  states  in  which  it  was  distributed  read  it.  This  high  percentage  of  respondents  reading 
the  handbook  may  be  facilitated  by  the  ease  with  which  beneficiaries  reported  understanding  it. 

When  asked  if  they  still  had  the  handbook  or  bulletin  in  their  possession,  75%  of  those 
respondents  who  said  they  got  it  indicated  that  they  did.  Most  beneficiaries  seem  to  be  filing  it 
in  a  place  where  they  can  retrieve  it  for  future  reference  when  they  have  a  question  or  decision 
that  requires  information.  If  beneficiaries  can  find  the  handbook  and  bulletin,  it  seems  like  it  is 
designed  well  enough  such  that  beneficiaries  can  understand  it  generally. 
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In  summary,  a  minority  of  beneficiaries  report  ever  getting  the  handbook  or  bulletin. 
However,  the  majority  of  beneficiaries  who  recall  receiving  it  indicate  that  they  read  all  or  parts 
of  it,  and  that  they  found  it  fairly  easy  to  understand.  Most  beneficiaries  seem  to  be  keeping  the 
handbook  for  future  reference. 

5.0  Conclusion 

For  each  of  the  NMEP  areas  that  we  examined,  some  but  not  all  of  the  evaluation 
measures  showed  that  the  1999  handbook  made  a  difference  in  the  outcome  being  measured. 
One  of  the  three  access  measures,  both  of  the  awareness  measures,  all  three  of  the  knowledge 
measures,  and  two  of  the  four  impact/use  measures  showed  a  significant  effect  of  having  read  the 
1999  handbook  on  the  outcome  measured.  In  all  8  of  the  12  regression  analyses  showed  that 
reading  the  1999  handbook  significantly  influenced  the  outcome,  and  these  8  analyses  were  in 
the  positive  direction. 

We  found  that  reading  the  1999  handbook  had  a  significant  effect  on  only  one  of  our 
three  access  questions —  whether  beneficiaries  tried  to  find  out  about  new  benefits  or  changes  in 
the  Medicare  program.  Beneficiaries  who  read  the  1999  handbook  in  the  five  states  that 
received  it  were  only  2.8  percentage  points  more  likely  to  have  tried  to  find  out  about  new 
benefits  or  changes.  Reading  the  1999  handbook  had  no  significant  effect  on  whether 
beneficiaries  tried  to  find  out  about  the  availability  and  benefits  of  Medicare  managed  care  plans 
and  what  Medigap  or  supplemental  insurance  policies  cover.  In  part,  these  questions  are  not  the 
best  measures  of  access  in  that  they  only  ask  beneficiaries  whether  they  tried  to  find  information. 
Better  measures  of  access  ask  whether  beneficiaries  actually  received  information.  In  addition, 
so  few  beneficiaries  tried  to  find  information  (5-12%)  that  follow-up  questions  had  sample  sizes 
to  small  for  meaningful  investigation  of  the  NMEP.  HCFA  should  continue  to  identify  better 
measures  of  access  to  information  that  more  effectively  measure  the  influence  of  the  NMEP. 

We  found  modest  effects  of  reading  the  1999  handbook  on  both  MCBS  questions  that  we 
used  to  measure  awareness.  Beneficiaries  who  read  the  1999  handbook  in  the  five  states  that 
received  it  were  approximately  8  percentage  points  more  likely  to  have  been  aware  of  Medicare 
MSAs  and  8.4  percentage  points  more  likely  to  have  been  aware  of  PPOs.  Only  16%  of 
beneficiaries  were  aware  of  MSAs  and  24%  were  aware  of  PPOs,  so  we  did  not  model  whether 
persons  were  aware  of  POSs  and  PSOs  as  even  fewer  beneficiaries  were  aware  of  these 
arrangements.  We  found  in  descriptive  analyses  that  only  9.5%  of  beneficiaries  had  heard  of  the 
buy-in  programs,  and  that  91 .7%  of  beneficiaries  were  unaware  that  this  type  of  help  was 
available.  These  results  suggest  that  reading  the  1999  handbook  has  potential  for  raising 
awareness  of  managed  care  arrangements  and  buy-in  programs. 

We  also  found  modest  effects  of  reading  the  1 999  handbook  for  the  NMEP  goal  of 
understanding.  We  found  approximately  5-7  percentage  point  gains  in  knowledge  scores  on 
three  separate  knowledge  indices  developed  for  measuring  beneficiary  understanding  of  different 
parts  of  the  Medicare  program.  All  three  of  these  findings  were  statistically  significant.  The 
index  on  which  beneficiaries  scored  the  largest  gain  (7.3  percentage  points)  was  the  8-item  quiz, 
the  majority  of  which  contained  questions  concerning  Medicare  managed  care  issues.  The  other 
two  knowledge  indices  for  which  reading  the  1999  handbook  in  the  five  states  had  an  effect  were 
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the  Perceived  Medicare  Knowledge  index  and  the  3 -item  quiz,  which  showed  percentage  point 
increases  of  5.9  and  4.8  respectively.  Clearly,  reading  the  1999  handbook  shows  potential  for 
increasing  knowledge  of  Medicare  beneficiaries.  A  consistent  set  of  questions  will  need  to  be 
administered  over  time  in  order  to  assess  the  longer  range  effects  of  the  NMEP. 

Use  of  well-constructed  knowledge  indices  may  have  an  advantage  in  that  they  minimize 
the  contribution  of  any  one  specific  question.  Other  MCBS  questions  designed  to  assess 
beneficiary  knowledge  about  plan  options  in  Medicare  +  Choice  that  were  not  included  in  the 
knowledge  indices  show  that  social  desirability  may  lead  to  upward  bias  in  knowledge  measures. 
Managed  care  arrangements  (like  PSOs  and  POSs)  that  beneficiaries  said  they  could  more 
frequently  join  in  Medicare  were  arrangements  they  understood  less  well.  These  findings 
suggest  the  importance  of  ongoing  development,  testing,  and  refinement  of  knowledge  indices  as 
an  important  way  to  measure  beneficiary  knowledge  and  limit  the  impact  of  social  desirability  in 
responses. 

We  also  measured  the  impact  and  use  of  the  1999  handbook  in  the  five  states,  finding 
significant  results  in  two  of  four  regression  models  we  analyzed.  Beneficiaries  who  read  the 
1999  handbook  were  16.7  percentage  points  more  likely  to  have  ever  used  the  1999  handbook  to 
find  information  about  health  plan  options,  and  12.2  percentage  points  more  likely  to  have 
reviewed  information  about  different  Medicare  health  plan  options.  These  results  were  the 
largest  percentage  point  gains  from  having  read  the  1999  handbook  in  all  of  our  analyses,  and 
demonstrate  that  persons  who  read  the  1999  handbook  tend  to  use  it  more. 

We  did  not  find  a  significant  effect  for  having  read  the  1999  handbook  in  satisfaction  or 
confidence  measures  we  analyzed.  Beneficiaries  were  no  more  likely  to  have  been  satisfied  with 
the  availability  of  information  about  the  Medicare  program  when  they  needed  it,  nor  were  they 
more  confident  that  they  could  choose  a  health  care  plan  that  best  matched  their  needs.  These 
results  suggest  that  beneficiaries  who  read  the  1999  handbook  may  find  that  their  choices  and 
options  are  more  complicated  and  numerous  than  previously  thought. 

Beneficiaries  who  get  the  1999  handbook  or  bulletin  materials  read  them  and  find  them 
fairly  easy  to  understand,  but  only  a  minority  of  beneficiaries  report  receiving  these  materials. 
Only  26.4%  of  all  respondents  reported  reading  all  or  parts  of  the  handbook.  We  found  that 
sizeable  majorities  of  beneficiaries  reported  the  materials  in  general  and  a  comparison  chart 
within  them  were  either  very  easy  or  somewhat  easy  to  understand,  but  almost  15%  of 
beneficiaries  said  the  opposite.  HCFA  should  continue  to  simplify  the  1999  handbook  to  reach 
more  beneficiaries.  Most  beneficiaries  who  get  the  1999  handbook  read  it,  and  most  keep  the 
1999  handbook  for  future  reference.  These  results  in  general  show  that  HCFA  should  continue 
to  fund  a  diverse  array  of  measures  to  educate  beneficiaries. 

In  general,  more  MCBS  questions  are  needed  to  better  measure  the  four  NMEP  goals  (see 
our  Phase  1  report,  Chapter  6).  We  found  some  survey  questions  in  the  MCBS  that  could  be 
used  to  evaluate  each  of  the  goals  of  the  NMEP,  though  some  were  better  than  others. 
Additional  survey  questions,  especially  questions  with  objective  answers,  are  needed  for 
knowledge  index  development. 
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Finally,  these  results  are  for  only  one  point  in  time.  In  the  future  results  from  the  same 
questions  and  analyses  may  show  increases  over  time,  or  they  may  not.  In  fact,  there  may  not  be 
large  increases  over  time  if  there  are  diminishing  returns  to  knowledge  improvement  as  a 
consequence  of  reading  the  1999  handbook  or  bulletin. 
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Appendix  A 
Exhibits  for  Results  of  Multivariate  Analyses 


A-l 


Exhibit  Al.  Logistic  Regression  for  Respondents  Who  Reported  Reading  Either  the 

Handbook  or  the  Bulletin 


(n=25,658,713) 


Medicare  Characteristic 

Coefficient 

Std.  Error 

Lived  in  handbook  states 

0.491** 

0.074 

Male 

0.165** 

0.052 

Age  65-75 

0.142 

0.093 

Over  Age  75 

0.003 

0.092 

Non-white 

0.002 

0.068 

Some  high  school 

0.327** 

0.079 

Some  college 

0.383** 

0.089 

Good  health 

-0.044 

0.058 

Fair/poor  health 

0.001 

0.077 

Proxy  responder 

-0.47 

0.25 

Person  makes  own  health 

0.66** 

0.14 

decisions 

Person  receives  heln  making 

1    "wlJ>V'll    IWVvlTVj    I1V1L/  UlUIYlllg 

0.56** 

0.15 

own  health  HecKion** 

Some  managed  care 

-0.102 

0.099 

Supplemental  insurance 

0.173* 

0.074 

Medicaid  eligibility 

0.127 

0.093 

$l-$499  in  charges 

0.082 

0.085 

$500-54,999  in  charges 

0.141 

0.078 

$5,000  +  in  charges 

-0.020 

0.083 

Perceived  Medicare  knowledge 

0.89** 

0.14 

Has  cable  TV 

0.009 

0.066 

Has  personal  computer 

0.085 

0.070 

Has  Internet  access 

0.016 

0.076 

Never  heard  of  Internet 

-0.02 

0.18 

Intercept 

-2.78** 

0.22 

Significant  at  p<0.05 


**  Significant  at  p<0.01 


Exhibit  A2.  Logistic  Regression  Results  Concerning  the  NMEP  Goal  of  Access  to 

Information 


BK7  BK15  BK19 

Info  on  benefits/changes      Info  on  managed  care  plans     Info  on  supplemental  plans 
(n=25,479.151)  (n=25,478,073)  (n=  17,323,382) 


Medicare  Characteristic 

Coefficient 

Std.  Error 

Coefficient 

Std.  Error 

Coefficient 

Std.  Error 

Read  handbook 

0.56** 

0.11 

0.25* 

0.12 

0.38** 

0.11 

Lived  in  handbook  states 

0.09 

0.15 

0.38 

0.20 

0.23 

0.18 

Interaction  term 

-0.05 

0.21 

-0.12 

0.20 

-0.27 

0.27 

Male 

-0.182* 

0.090 

-0.114 

0.082 

-0.02 

0.08 

Age  65-75 

-0.41** 

0.15 

-0.22 

0.17 

-0.38 

0.20 

Over  Age  75 

-0.74** 

0.16 

-0.46** 

0.16 

-0.65** 

0.21 

Non-white 

0.16 

0.13 

-0.54** 

0.14 

-0.19 

0.16 

Some  hif?h  school 

0.47** 

0.15 

0.21 

0.14 

-0.02 

0.12 

Some  college 

0.81** 

0.16 

0.54** 

0.16 

0.07 

0.13 

OooH  health 

0.06 

0.11 

0.02 

0.11 

0.01 

0.09 

Fair/noor  health 

0.30* 

0.12 

0.21 

0.14 

0.00 

0.13 

Proxy  responder 

-0.35 

0.47 

-0.50 

0.48 

0.65 

0.36 

Pprcnn  malrpc  own  hpalth 

0.40 

0.31 

-0.09 

0.25 

-0.33 

0.20 

decisions 

PprQAn  rpppive^  hpln 

0.48 

0.29 

-0.03 

0.24 

-0.19 

0.19 

maVino  nu/n  hpaltn 

UlaMIIU  uwu  lltaiui 

decisions 

Some  managed  care 

0.18 

0.19 

1.58** 

0.16 

-0.32 

0.21 

Supplemental  insurance 

-0.12 

0.15 

-0.37** 

0.13 

-0.16 

0.28 

Medicaid  eligibility 

-0.36* 

0.18 

-0.27 

0.17 

-0.18 

0.38 

$l-$499  in  charges 

0.36 

0.18 

0.56** 

0.17 

-0.04 

0.18 

$500-54,999  in  charges 

0.42* 

0.19 

0.48** 

0.17 

0.14 

0.16 

$5,000  +  in  charges 

0.65** 

0.19 

0.39* 

0.19 

-0.03 

0.19 

Perceived  Medicare 

0.44* 

0.20 

0.82** 

0.22 

0.39* 

0.17 

knowledge 

0.09 

Has  cable  TV 

-0.061 

0.089 

-0.19 

0.13 

-0.06 

Has  personal  computer 

-0.04 

0.13 

0.13 

0.13 

-0.05 

0.15 

Has  Internet  access 

0.20 

0.13 

0.08 

0.15 

0.18 

0.13 

Never  heard  of  Internet 

-0.85* 

0.42 

-0.58 

0.60 

-0.69 

0.44 

Intercept 

-4.12** 

0.39 

-3.65** 

0.37 

-1.50** 

0.37 

Significant  at  p<0.05 


Significant  at  p<0.01 


Exhibit  A3.  Logistic  Regression  Results  Concerning  the  NMEP  Goal  of  Awareness 

BK64  BK66 

Ever  heard  of  a  MSA  Ever  heard  of  a  PPO 


(n=25,487,734)  (n=25,484,653) 


Medicare  Characteristic 

Coefficient 

Std  error 

Coefficient 

Std.  Error 

Read  handbook 

.509»* 

.076** 

.493** 

.058 

Lived  in  handbook  states 

.02 

.11 

.10 

.13 

Interaction  term 

.10 

.15 

.03 

.12 

Male 

.284** 

.061 

.203** 

.052 

Age  65-75 

.22 

.12 

-.253** 

.096 

Over  Age  75 

.06 

.12 

-.82** 

.11 

Non-white 

-.50** 

.11 

-.530** 

.094 

Some  high  school 

.40** 

.12 

.56** 

.11 

Some  college 

1.00** 

.13 

1.27** 

.11 

Good  health 

-.069 

.071 

-.190** 

.067 

Fair/poor  health 

-.225** 

.084 

-.238** 

.076 

Proxy  responder 

-.33 

.38 

-.16 

.31 

Person  makes  own  health 

.62** 

.15 

.13 

.17 

decisions 

Person  receives  helD  making 

.37* 

.17 

.08 

.19 

own  health  decisions 

Some  managed  care 

.19 

.12 

.320** 

.097 

Supplemental  insurance 

.36** 

.10 

.560** 

.084 

Medicaid  eligibility 

-.08 

.15 

-.16 

.13 

$l-$499  in  charges 

.13 

.12 

-.013 

.099 

$500-$4,999  in  charges 

.22 

.12 

-.02 

.10 

$5,000  +  in  charges 

.02 

.13 

.02 

.11 

Perceived  Medicare 

1.35** 

.13 

1.17** 

.14 

knowledge 

Has  cable  TV 

-.104 

.065 

-.117 

.062 

Has  personal  computer 

.370** 

.078 

.417** 

.086 

Has  Internet  access 

.379** 

.082 

.383** 

.086 

Never  heard  of  Internet 

-.72 

.37 

-.30 

.28 

Intercept 

-4.32** 

.21 

-2.89** 

.22 

Significant  at  p<0.05 


**      Significant  at  p<0.01 


Exhibit  A4.  OLS  Regression  Results  Concerning  the  NMEP  Goal  of  Understanding 


8-item  quiz        Perceived  Medicare  Knowledge         3-item  quiz 

(n=25,603,774)  (n=25,489,087)  (n=25,605,588) 


Medicare 
Characteristic 

Coefficient 

Std.  Error 

Coefficient 

Std.  Error 

Coefficient 

Std.  Error 

Read  handbook 

0.1004** 

0.0081 

0.0418** 

0.0078 

0.051 

0.010 

Lived  in  handbook 

0.077** 

0.012 

0.001 

0.012 

0.010 

0.013 

states 

Interaction  term 

-0.027* 

0.013 

0.017 

0.012 

-0.004 

0.016 

Male 

0.0144* 

0.0057 

.0059 

.0043 

-0.0095 

0.0062 

Age  65-75 

0.034* 

0.012 

0.0312** 

0.0093 

0.070** 

0.011 

Over  Age  75 

-0.007 

0.011 

0.0130 

0.0097 

0.027* 

0.012 

Non-white 

-0.057** 

0.012 

-0.066** 

0.010 

-0.089** 

0.011 

Some  high  school 

0.0619** 

0.0086 

0.0587** 

0.0073 

0.0436** 

0.0099 

Some  college 

0.102** 

0.011 

0.0897** 

0.0092 

0.093** 

0.011 

Good  health 

-0.0207** 

0.0080 

-0.0190** 

0.0056 

-0.0023 

0.0090 

Fair/poor  health 

-0.0449** 

0.0088 

-0.0447** 

0.0073 

-0.0095 

0.0095 

Proxy  responder 

0.008 

0.028 

0.004 

0.022 

-0.000 

0.032 

Person  makes  own 

1    wl  JV.M  1    11  lUIvvO    V  *  *  IB 

0.134** 

0.013 

0.100** 

0.013 

0.140** 

0.016 

health  decisions 

Person  receives  hclo 

0.100** 

0.014 

0.058** 

0.014 

0.103** 

0.015 

makinp  own  health 

decisions 

0.014 

Some  managed  care 

0.182** 

0.013 

0.127** 

0.011 

0.126** 

Supplemental  insurance 

0.038** 

0.011 

0.0650** 

0.0095 

0.0707** 

0.0099 

Medicaid  eligibility 

-0.031** 

0.012 

-0.0166 

0.0096 

-0.001 

0.012 

$  1  -$499  in  charges 

0.0396** 

0.0097 

0.037** 

0.010 

0.072** 

0.012 

$500-54,999  in  charges 

0.0635** 

0.0093 

0.0627** 

0.0098 

0.116** 

0.011 

$5,000  +  in  charges 

0.060** 

0.010 

0.071** 

0.011 

0.123** 

0.011 

Has  cable  TV 

0.0222** 

0.0071 

0.0312** 

0.0058 

0.0331** 

0.0080 

Has  personal  computer 

0.0308** 

0.0085 

0.0218** 

0.0075 

0.017 

0.011 

Has  Internet  access 

0.0342** 

0.0084 

0.0356** 

0.0076 

0.034** 

0.010 

Never  heard  of  Internet 

-0.125** 

0.021 

-0.064** 

0.023 

-0.080** 

0.023 

Intercept 

0.24 

0.023 

0.233** 

0.022 

0.202 

0.024 

Significant  at  p<0.05 


**      Significant  at  p<0.01 


Exhibit  A5.  Logistic  and  Ordered  Logit  Regression  Results  Concerning  the  NMEP  Goal  of  Impact/Use 


Used  Information  Used  book/bulletin  Satisfied  with  Information  Confidence  in  choices 

(n-14,551)  (n=14,551)  Availability  (n=14,551) 

(n=14,551) 


Medicare  Characteristic 

Coefficient 

Std  error 

Coefficient 

Ctrl  arrAr 

V  UC  1  1  ICIl  II I 

^tH  prrnr 

O LU  CI  1  Ul 

C  f\of  fir  ion  1 

^tH  prrnr 

ulU  CI  I  Ul 

Read  handbook 

1.822** 

0.407 

1.195** 

0.091 

0.278" 

U.  1  Jo 

A  A/1  A 

U.UoU 

Lived  in  handbook  states 

-0.350 

A  £.11 

U.UD4 

n  in 

U.  1  j  \ 

n  167 

ft  H? 

n  i?6 

Interaction  term 

i  ac7 

u.ouz 

0. 156 

0.032 

0.186 

0.048 

0.120 

Male 

0.099 

A  1  C  A 

0. 154 

-U.U04 

U.UOj 

U.I  J7 

0  116** 

U.  1  IU 

n  mo 

Age  65-75 

A  A  no* * 
-U.4UV 

f\  ion 

0  I9Q 

0.329** 

0.1 1 1 

0.021 

0.068 

Over  Age  75 

-0.638* 

0.249 

0.062 

0.133 

0.415** 

0.113 

-0.106 

0.072 

Non-white 

0.077 

0.241 

0.021 

0.121 

-0.010 

0.080 

-0.049 

0.066 

Some  high  school 

0.110 

0.222 

0.125 

0.121 

-0.022 

0.088 

0.260** 

0.064 

Some  college 

0.150 

0.252 

0.332** 

0.130 

0.050 

0.095 

0.400** 

0.085 

Good  health 

-0.180 

0.154 

-0.006 

0.082 

-0.140* 

0.062 

-0.177** 

0.048 

Fair/poor  health 

0.109 

0.149 

-0.015 

0.090 

-0.342** 

0.084 

-0.317** 

0.051 

Proxy  responder 

-0.593 

0.698 

-1.736** 

0.517 

-0.409 

0.323 

-0.147 

0.208 

Person  makes  own  health  decisions 

0.627 

0.436 

0.276 

0.228 

0.068 

0.141 

1.248** 

0.134 

Person  receives  help  making  own  health 

0.563 

0.461 

0.005 

0.223 

0.001 

0.136 

0.741** 

0.122 

decisions 

Some  managed  care 

0.504* 

A  "IA  C 

0.246 

f\  1  *)A 

U.  1  JL 

0.1 13 

0.269** 

0.081 

Supplemental  insurance 

-0.301 

0.174 

0.119 

0.098 

0.004 

0.090 

0.279** 

0.057 

Medicaid  eligibility 

-0.100 

0.202 

-0.029 

0.116 

0.234 

0.121 

-0.084 

0.068 

$  1  -$499  in  charges 

0.633** 

0.229 

0.126 

0.130 

-0.143 

0.095 

-0.129 

0.081 

$500-54,999  in  charges 

0.762** 

0.223 

•  0.093 

0.105 

-0.047 

0.102 

-0.128 

0.081 

$5,000  +  in  charges 

0.741** 

0.270 

-0.028 

0.123 

-0.102 

0.106 

-0.103 

0.079 

Perceived  Medicare  knowledge 

0.688* 

0.316 

1.163** 

0.146 

1.545** 

0.150 

1.672** 

0.134 

Has  cable  TV 

0.035 

0.153 

-0.1157 

0.077 

0.062 

0.063 

0.038 

0.043 

Has  personal  computer 

0.053 

0.179 

0.095 

0.085 

0.046 

0.077 

0.261** 

0.058 

Has  Internet  access 

0.157 

0.169 

0.363** 

0.097 

-0.025 

0.093 

0.038 

0.064 

-2.164* 

1.025 

-0.992** 

0.369 

-0.133 

0.139 

-0.280* 

0.126 

Exhibit  A5.  Logistic  and  Ordered  Logit  Regression  Results  Concerning  the  NMEP  Goal  of  Impact/Use 


Used  Information  Used  book/bulletin  Satisfied  with  Information  Confidence  in  choices 

(n=14,551)  (n=14,551)  Availability  (n=14,551) 

(n=14,551) 


Medicare  Characteristic 

Coefficient 

Std  error 

Coefficient 

Std  error 

Coefficient 

Std  error 

Coefficient 

Std  error 

Intercept  1 

-5.331** 

0.759 

-3.710** 

0.333 

-3.293** 

0.220 

-5.211** 

0.209 

Intercept  2 

1.519** 

0.208 

-2.982** 

0.210 

Intercept  3 

3.064** 

0.241 

-1.363** 

0.188 

Intercept  4 

0.003 

0.171 

Significant  at  p<0.05 


Significant  at  p<0.01 


Appendix  B 
Bar  Charts  for  NMEP  Goal  on  Access 


100%  - 

Exhibit  Bl 

Question  BK  3:  In  the  past  year,  have  you  tried  to  find  out  how 
much  you  needed  to  pay  for  a  particular  medical  service? 

93.5% 

80%  . 

60%  - 

40% 

20%  - 

6.2% 

0% 

 ^  1  

Yes 

No 

Exhibit  B2 

Dependent  Variables  Used  in  Multivariate  Analyses 
Question  BK  7:  In  the  past  year,  have  you  tried  to  find  information 
about  any  new  benefits  or  changes  in  the  Medicare  program? 


100V.  . 

Exhibit  B3 

Question  BK  11:  In  the  past  year,  have  you  tried  to  find 
information  about  what  medical  services  Medicare  covers  and 
does  not  cover? 

91.8% 

80%  - 

60V.  . 

40%  . 

20%  . 

8.0% 

0% 

Yes 

No 

Exhibit  B4 

Dependent  Variables  Used  in  Multivariate  Analyses 
Question  BK  15:  In  the  past  year,  have  you  tried  to  find  out 
about  the  availability  and  benefits  of  Medicare  managed  care 
plans,  such  as  HMOs? 

100%  . 

94.4% 

80%  . 

60%  . 

40%  . 

20%  . 

0% 

5.6% 

Yes 

No 

100% 

Exhibit  B5 

Dependent  Variables  Used  in  Multivariate  Analyses 
Question  BK  19:  In  the  past  year,  have  you  tried  to  find 
information  about  what  your  Medigap  or  supplemental 
insurance  policy  covers? 

88.1% 

80%  . 

60%  . 

40%  . 

20%  . 
0% 

11.9% 

 (  

1  1 

Yes 

No 

Exhibit  B6 

Question  BK  23:  In  the  past  year,  have  you  tried  to  find 
information  about  your  Medicaid  plan,  such  as  how  it  works 
with  Medicare? 

100% 

87.7% 

80%  . 

60%  . 

40%  . 

20%  . 
0% 

11.4% 

0.8% 

Yes                                 No                        Not  on  Medicaid 

Exhibit  B7 

Question  BK  52:  You  can't  really  trust  what  you  see  on  television 
or  read  in  the  newspapers  about  Medicare.  Is  that  mostly  true  for 
you,  mostly  false  for  you,  or  neither  true  nor  false? 


100% 


60%  . 


20%  . 


32.6% 

23.1% 


I 


2.1% 


Mostly  true  Mostly  false        Neither  true  nor  false        Don't  know 


Appendix  C 
Bar  Charts  for  NMEP  Goal  on  Awareness 


c-i 


Exhibit  CI 

Dependent  Variables  Used  in  Multivariate  Analyses 
Question  BK  64:  Before  now,  had  you  ever  heard  of  a  Medical 
Savings  Account? 

100% 

83.6% 

80%  . 

60%  . 

40%  . 

20%  . 

16.4% 

0% 

Yes 

No 

Exhibit  C2 

Dependent  Variables  Used  in  Multivariate  Analyses 
Question  BK  66:  Have  you  ever  heard  of  a  Preferred  Provider 
Organization,  or  PPO? 

ioo%  n 

80%  - 

76.0% 

60%  - 

40% . 

20%  - 

24.0% 

0% 

Yes  No 

Exhibit  C3 

Question  BN  8:  Have  you  ever  heard  of  any  of  the  Medicare  Buy- 
In  Programs? 

100%  . 

90.0% 

80%  . 

60%  . 

40%  . 

20%  . 

9.5% 

0% 

Yes 

No 

Exhibit  C4 

Question  BN  9:  Before  now,  were  you  aware  that  benefits  (from 

Medicare  Buy-In  Programs)  were  available? 

100% 

91.7% 

80%  . 

60%  . 

40%  - 

20%  - 

8.3% 

10.0% 

0% 

Yes 

No 

Not  applicable 

Appendix  D 
Bar  Charts  for  NMEP  Goal  on  Understanding 


Exhibit  Dl 

Question  BK  1 

:  How  much  do  you  think  you  know 

about  the 

\Ip  f\ir 

,'KUIl 

sic  jjiugiam 

? 

100% 

80%  . 

60%  - 

40%  . 
20%  - 

9.5% 

24.9% 

1 

29.0% 

1 

21.1% 

15.5% 

0% 

■  , 

1 

Just  about 

Most  of  what 

Some  of  what 

Little  of  what 

Almost  none  of 

everything  you 

you  need  to 

you  need  to 

you  need  to 

what  you  need 

need  to  know 

know 

know 

know 

to  know 

Exhibit  D2 

Question  BK  65:  How  well  do  you  understand  Medical 

Savings  Accounts? 

100%  - 

80%  - 

60%  - 

40%  - 

39.7% 

30.2% 

rnjjn  20.7% 

20%  - 

■  111 

0%  . 

Very  well              Fairly  well           Not  too  well             Not  at  all 

Exhibit  D3 

Question  BK  67:  How  well  do  you  understand  PPOs? 

100% 

80%  . 
60%  . 

40%  . 

33.2% 

29.6% 

21.6% 

20%  - 
0% 

■  III 

Very  wll                 Fairly  mil               Not  too  well                Not  at  all 

Exhibit  D4 

Question  BK  70:  How  well  do  you  understand  PSOs? 


100%  n 


80% 


60%  . 


40%  . 


20% 


0% 


35.0% 


I.S% 


29.1% 


24.4% 


Very  well  Fairly  well  Not  too  well  Not  at  all 


Exhibit  D5 

Question  BK  73:  How  well  do  you  understand  POSs? 

100%  , 

80%  . 

60%  - 
40V.  - 
20%  - 

36.3% 

26.4%  27.7% 

m    1    1  1 

0%  . 

Very  well                    Fairly  well                  Not  too  well                  Not  at  all 

Exhibit  D6 

Question  BK  68:  Can  people  on  Medicare  join  PPO  plans? 

100%  , 
80%  . 
60%- 

55.9% 

44.1% 

40%  - 
20%  - 
0%. 

Yes  No 

Exhibit  D7 

Question  BK  71:  Can  people  on  Medicare  join  PSO  plans? 

100% 

80%  . 

60%  . 

59.4% 

40%  . 

40.6% 

20%  . 

0% 

Yes 

No 

Exhibit  D8 

Question  BK  74:  Can  people  on  Medicare  join  HMO  POS 
plans? 


100%  , 


80% 


60% 


40% 


20% 


0% 


61.7% 

38.3% 

1 

Yes 


No 


Exhibit  D9 

Question  BK  43:  Most  people  covered  by  Medicare  can  select 
among  different  health  plan  options  within 
Medicare  -True  or  False? 

100% 

80%  . 

60%  . 

47.3% 

52.7% 

40%  - 

20%  - 

0% 

Correct 

Incorrect 

Exhibit  D10 

Question  BK  44:  Medicare  without  a  supplemental  insurance 
policy  pays  for  all  of  your  health  care  expenses  -  True  or  False? 

100% 

80%  . 

77.3% 

60%  . 

40%  - 

20%  - 

22.7% 

0% 

 ,  ^^^^^^ 

Correct 

Incorrect 

Exhibit  Dll 

Question  BK  45:  If  you  are  happy  with  the  way  you  currently 
receive  health  care,  you  do  not  have  to  make  changes  in  the  way 
you  get  your  Medicare  services  -  True  or  False? 

100%  . 

80% 

72.2% 

60%  . 

40%  . 

27.8% 

20%  - 

0% 

Correct 

Incorrect 

Exhibit  D12 

Question  BK  46:  The  Medicare  program  has  recently  begun  to 
offer  more  information  and  help  in  order  to  answer  your 
Medicare  questions  -  True  or  False? 

100% 


80%  . 


Correct  Incorrect 


Exhibit  D13 

Question  BK  47:  People  can  report  complaints  to 

Medicare  about  their  Medicare  managed  care  plans  (HMOs)  or 

supplemental  plans  if  they  are  not  satisfied  with  them  -  True  or 

False? 

100%  - 

80%  - 

60% . 

57.8% 

40%  . 

42.2% 

20%  - 

0% 

Correct 

Incorrect 

Exhibit  D14 

Question  BK  48:  If  someone  joins  a  Medicare  managed  care  plan 
(HMO)  that  covers  people  on  Medicare,  they  have  limited 
choices  about  which  doctors  they  can  see  -  True  or  False? 

100%-, 


80% 


Correct  Incorrect 


Exhibit  D15 

Question  BK  49:  If  someone  joins  a  Medicare  managed  care  plan 
(HMO)  that  covers  people  on  Medicare,  they  can  change  or  drop 
the  plan  and  still  be  covered  by  Medicare  - 
True  or  False? 

100% 

80%  . 


40%  . 
20%  . 
0%  . . 


Exhibit  D16 

Question  BK  50:  Medicare  managed  care  plans  (HMOs)  that 
cover  people  on  Medicare  often  cover  more  health  services,  like 
prescribed  medicines,  than  Medicare  without  a  supplemental 
policy  -  True  or  False? 

100%  - 
80%  - 
60%  . 
40%  . 
20%  . 


Exhibit  D17 

Question  BN  1:  How  much  do  you  feel  you  know  about  what 
medical  services  Medicare  covers  or  does  not  cover? 


100% 


80%  . 


60% 


40% 


20% 


0% 


28.9%  28.2% 


■  ■  i 

Just  about  all         Most  of  Some  of  A  little  Almost  none 


Exhibit  D18 

Question  BN  2:  How  much  do  you  feel  you  know  about  how 
much  you  have  to  pay  for  medical  services? 


100% 


80%  . 


60% 


40% 


20% 


0% 


29.1% 


24.2% 

1       l       I  I 


12.2% 


Just  about  all         Most  of  Some  of  A  little  Almost  none 


Exhibit  D19 

Question  BN  3:  How  much  do  you  feel  you  know  about 
supplemental  or  Medigap  insurance,  such  as  what  it  covers  or 
how  it  works  with  Medicare  to  pay  medical  claims? 

100% 
80%  . 
60%  . 
40%  . 
20% 
0% 


24.6%  24.4% 

.    i5.5%  is.2%  mm 

■  I  1  ■  1 

Just  about  all         Most  of  Some  of             A  little  Almost  none 


100%  - 

Exhibit  D20 

Question  BN  4:  How  much  do  you  feel  you  know  about  the 
availability  and  benefits  of  Medicare  managed  care  plans? 

80%  - 

60%  . 

40%  - 

40.8% 

20%  . 
0%  . 

14.3%              15.'%  17-3% 

Li  ■  ■  ■ 

1 

Just  about  all         Most  of           Some  of            A  little 

Almost  none 

Exhibit  D21 

Question  BN  5:  How  much  do  you  feel  you  know  about  choosing 
or  finding  a  doctor  or  health  care  provider? 

100% 

80%  - 

60%  - 

40%  - 
20%  . 
0%  - 

33.4V. 

27.9%  p~— 1 

1      1  19.6% 

III.- 

Just  about  all         Most  of           Some  of            A  little          Almost  none 

Exhibit  D22 

Question  BN  16:  Medicare  covers  colorectal  cancer  screening  - 
True  or  False? 

100%  _ 


80%  . 


Correct  Incorrect 


Exhibit  D23 

Question  BN  18:  Medigap  or  supplemental  insurance  is  the  same 
as  a  Medicare  managed  care  plan  - 
True  or  False? 

100% ., 


40% 


20% 


0% 


56.0% 

43.9% 

Correct 

Incorrect 

Exhibit  D24 

Question  BN  19:  Medicare  covers  an  annual  flu  shot  -  True  or 
False? 

100% 

84.0% 

80%  - 

60%  . 

40%  . 

20%  - 

15.9% 

0% 

Correct 

Incorrect 

Appendix  E 
Bar  Charts  for  NMEP  Goal  on  Impact/Use 


E-l 


100%  , 

Exhibit  El 

Question  BK  30:  Would  you  say  you  have  read  this 
(book/bulletin)  throughly,  that  you  have  read  parts  of  it,  or  that 
you  haven't  read  it  at  all? 

80%  . 

63  8% 

60%  . 

40%  . 

i 

20%  . 

19.3% 

H   hi  H 

0% 

Read  it  thoroughly      Read  parts  of  it     Have  not  read  it  at    Said  didn't  receive 

all  book 

Exhibit  E2 

Question  BK  32:  Have  you  ever  used  this  (book/bulletin)  to  find 
information  about  health  plan  options  available  to  you,  such  as 
Medicare  managed  care  plans,  HMO  or  Supplemental  plans? 

100% 

91.6% 

80%  . 

60%  - 

40%  . 

20%  . 

8.4% 

0% 

 H^^^B   1  

Yes 

No 

Exhibit  E3 

"Dependent  Variables  Used  in  Multivariate  Analyses" 
Question  BK  58:  Have  you  reviewed  information  about  different 
Medicare  health  plan  options? 

100% 

87.7% 

80%  . 

60%  . 

40%  . 

20%  - 

12.3% 

0% 

 ^  1  

1  1 

Yes 

No 

Exhibit  E4 

"Dependent  Variables  Used  in  Multivariate  Analyses" 

Question  BK  2:  How  satisfied  are  you  in  general  with  the 

availablitly  of  information  about  the  Medicare  program  when  you 

need  it? 

100%  , 

81.4% 

80% . 

60%. 

40%  - 

20% . 

11.6% 

0% 

r-— 5.4% 

1.6% 

Very  satisfied              Satisfied                Unsatisfied           Very  unsatisfied 

Exhibit  E5 

"Dependent  Variables  Used  in  Multivariate  Analyses" 
Question  BK  62:  How  confident  are  you  that  you  could  choose  a 
health  care  plan  that  best  matches  your  needs? 


100% 


80%  . 


60% 


40% 


20% 


0% 


34.4% 


29.2% 

18.6% 

■  II- 


11.2% 

Extremely       Very  confident      Moderately  Slightly         Not  confident 

confident  confident  confident 


Appendix  F 

Bar  Charts  for  Direct  Evaluation  of  NMEP  Materials 


F-l 


Exhibit  Fl 

Question  BK  28:  Did  you  receive  a  copy  of  this  (book/bulletin), 
called  Medicare  &  You,  in  the  mail  sometime  in  the  past  year? 


80% 


20% 


60.6% 

39.4% 

Yes  No 


Exhibit  F2 

Question  BK  33:  How  easy  to  understand  did  you  find  (this 
book/this  bulletin/the  parts  you  looked  at)? 


100% 


80%  . 


60%  54.6% 


40% 


20%  - 


0% 


30.5% 


12.5% 

2.4% 


Very  easy  Somewhat  easy      Somewhat  difficult       Very  difficult 


Exhibit  F3 
Question  BK  37:  How  easy  did  you  find 
the  Comparison  Chart  to  understand? 

100%  , 


80%  . 


Very  easy  Somewhat  easy      Somewhat  difficult        Very  difficult 


Exhibit  F4 
Question  BK  40:  Do  you  still  have  the 
Medicare  &  You  (book/bulletin)? 

100% 

80%  . 

75.4% 

60%  . 

40%  . 

20%  - 

24.6% 

0% 

Yes 

No 

Appendix  G 

Bar  Charts  for  Independent  Variables  Used  in  Multivariate 

Analysis 


G-l 


Exhibit  Gl 

Independent  Variables  Used  in  Multivariate  Analysis 
Persons  Who  Read  the  Handbook  or  Bulletin 


100%  , 


80%  - 

73.6% 

60%  . 

40%  . 

26.4% 

20%  . 

0% 

i 

Read  book/bulletin  Did  not  read  book/bulletin 


Exhibit  G2 

Independent  Variables  Used  in  Multivariate  Analysis 
Does  Beneficiary  Live  In  One  of  the  Five  States  that  Received  the 
M  &  Y  Handbook? 

100  - 

86.7 

80  . 

60  . 

40  . 

20  - 

13.3 

0 

'  1 

In  5  states 

Not  in  5  states 

Exhibit  G3 

Independent  Variables  Used  in  Multivariate  Analysis 
Gender 


55.7% 

44.3% 

Female  Male 


100% 

Exhibit  G4 

Independent  Variables  Used  in  Multivariate  Analysis 
Age  Group 

80%  . 

60%  . 

50.4% 

40%  . 

36.9% 

20%  . 
0% 

12.6% 

1  

Under  65 

65-75 

Over  75 

Exhibit  G5 

Independent  Variables  Used  in  Multivariate  Analysis 
Race 


100% 


80% 


60% 


40% 


20%  . 


0% 


82.0% 


White 


Non-white 


Exhibit  G6 

Independent  Variables  Used  in  Multivariate  Analysis 
Education 


100% 


80% 


60%  . 


40%  . 


20% 


0% 


49.4% 

30.2% 

20.4% 

8th  grade  or  leu  Some/completed  HS         Some/completed  college 


Exhibit  G7 

Independent  Variables  Used  in  Multivariate  Analysis 
Self-Reported  Health  Status 

100%  , 

80%  . 


Excellent/Very  good  Good  Fair/Poor 


Exhibit  G8 

Independent  Variables  Used  in  Multivariate  Analysis 
Proxy  Respondent  Answered  MCBS  Survey  Questions 


Has  a  proiy  Does  not  have  a  proxy 


Exhibit  G9 

Independent  Variables  Used  in  Multivariate  Analysis 
Does  Beneficiary  Need  Help  in  Making  Health  Insurance 
Decisions? 


100% 


80% 


60%  - 


40% 


20% 


0% 


62.3% 


22.9% 

1  1  

14.1% 

Makes  decisions  on  own  Gets  help  on  decisions 


Someone  else  makes 
decisions 


Exhibit  G10 

Independent  Variables  Used  in  Multivariate  Analysis 
Any  Managed  Care  Enrollment 

100% 

86.7% 

80%  . 

60%  . 

40%  . 

20%  - 

13.3% 

0% 

Enrolled  at  some  time 

No  enrollment 

Exhibit  Gil 

Independent  Variables  Used  in  Multivariate  Analysis 
Type  of  Health  Insurance 

100%  . 

80%  - 

60%  . 

62.3% 

40%  - 

20%  - 

21.7% 

15.9% 

0% 

Medicare  only             Medicare  and  private        Medicare  and  Medicaid 

100% 


80% 


60% 


40% 


20% 


0% 


Exhibit  G12 

Independent  Variables  Used  in  Multivariate  Analysis 
Amount  of  Total  Medicare  Charges 


20.8% 


SO 


36.5% 


20.2% 


22.5% 


SI -$499 


$500-54,999        $5,000  or  more 


Exhibit  G13 

Independent  Variables  Used  in  Multivariate  Analysis 
Whether  Beneficiary  Has  Cable  TV  in  Their  Home 

100%  - 

80%  . 

71.0% 

60%  . 

40%  - 

29.0% 

20%  - 

0% 

Yes 

No 

Exhibit  G14 

Independent  Variables  Used  in  Multivariate  Analysis 
Whether  Beneficiary  Has  a  Personal  Computer 
in  Their  Home 

100%  , 

80%  . 

77.8% 

60%  . 

40%  - 

20%  . 

22.2% 

0%  . 

Yes  No 

Exhibit  G15 

Independent  Variables  Used  in  Multivariate  Analysis 
Whether  Beneficiary  Has  Access  to  the  Internet 


100%  , 


Appendix  H 

Sample  Sizes  for  Bar  Charts  Presented  in  Appendices  B-G 


V 


H-1 


Sample  Sizes  for  Bar  Charts  Presented  in  Appendices  B-G 


Exhibit 

Sample  size 

Bl 

30,105,201 

B2 

30,056,367 

B3 

30,105,201 

B4 

30,055,322 

B5 

19,896,620 

B6 

30,105,201 

B7 

30,105,201 

CI 

30,087,808 

C2 

30,084,343 

C3 

28,640,986 

C4 

28,640,986 

Dl 

29,917,640 

D2 

4,907,975 

D3 

7,222,037 

D4 

2,442,400 

D5 

1,578,905 

D6 

7,226,544 

D7 

1,582,545 

D8 

2,443,881 

D9 

30,069,559 

DIO 

30,067,220 

Dll 

30,067,220 

Dl2 

30,067,220 

Dl3 

30,067,220 

Dl4 

30,067,220 

Dl5 

30,067,220 

Dl6 

30,066,053 

Dl7 

28,640,986 

Dl8 

28,640,986 

Dl9 

28,640,986 

D20 

28,640,986 

D2l 

28,640,986 

D22 

28,640,986 

D23 

28,640,986 

D24 

28,640,986 

El 

30,105,201 

E2 

10,881,368 

E3 

27,011,316 

E4 

26,413,420 

E5 

26,203,604 

Exhibit 

Sample  size 

Fl 

30,079,354 

F2 

8,014,428 

F3 

648,650 

F4 

10,884,499 

Gl 

30,105,201 

G2 

30,105,201 

G3 

30,089,739 

G4 

30,089,739 

G5 

29,930,958 

G6 

30,089,739 

G7 

30,038,927 

G8 

30,089,739 

G9 

30,105,201 

G10 

30,089,739 

Gil 

30,089,739 

G12 

30,089,739 

G13 

28,600,372 

G14 

28,603,672 

G15 

28,612,206 
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